{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<i>Copyright (c) Microsoft Corporation. All rights reserved.</i>\n",
    "\n",
    "<i>Licensed under the MIT License.</i>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing different Hyperparameters and Benchmarking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we'll cover how to test different hyperparameters for a particular dataset and how to benchmark different parameters across a group of datasets.\n",
    "\n",
    "For an example of how to scale up with remote GPU clusters on Azure Machine Learning, please view [24_exploring_hyperparameters_on_azureml.ipynb](24_exploring_hyperparameters_on_azureml.ipynb).\n",
    "## Table of Contents\n",
    "\n",
    "* [Testing hyperparameters](#hyperparam)\n",
    "  * [Using Python](#python)\n",
    "  * [Using the CLI](#cli)\n",
    "  * [Visualizing the results](#visualize)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing hyperparameters  <a name=\"hyperparam\"></a>\n",
    "\n",
    "Let's say we want to learn more about __how different learning rates and different image sizes affect our model's accuracy when restricted to 10 epochs__, and we want to build an experiment to test out these hyperparameters. We also want to try these parameters out on two different variations of the dataset - one where the images are kept raw (maybe there is a watermark on the image) and one where the images have been altered (the same dataset where there was some attempt to remove the watermark).\n",
    "\n",
    "In this notebook, we'll walk through how we use the Parameter Sweeper module with the following:\n",
    "\n",
    "- use python to perform this experiment\n",
    "- use the CLI to perform this experiment\n",
    "- evalute the results using Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensure edits to libraries are loaded and plotting is shown in the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by importing the utilities we need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0.48'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sys\n",
    "import scrapbook as sb\n",
    "import fastai\n",
    "sys.path.append(\"../../\")\n",
    "from utils_cv.classification.data import Urls\n",
    "from utils_cv.common.data import unzip_url\n",
    "from utils_cv.classification.parameter_sweeper import ParameterSweeper, clean_sweeper_df, plot_sweeper_df\n",
    "\n",
    "fastai.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use the Parameter Sweeper tool for single label classification, we'll need to make sure that the data is stored such that images are sorted into their classes inside of a subfolder. In this notebook, we'll use the Fridge Objects dataset, which is already stored in the correct format. We also want to use the Fridge Objects Watermarked dataset. We want to see whether the original images (which are watermarked) will perform just as well as the non-watermarked images.\n",
    "\n",
    "Define some parameters we will use in this notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "DATA = [unzip_url(Urls.fridge_objects_path, exist_ok=True), unzip_url(Urls.fridge_objects_watermark_path, exist_ok=True)]\n",
    "REPS = 3\n",
    "LEARNING_RATES = [1e-3, 1e-4, 1e-5]\n",
    "IM_SIZES = [299, 499]\n",
    "EPOCHS = [10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Python <a name=\"python\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by creating the Parameter Sweeper object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "sweeper = ParameterSweeper()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we start testing, it's a good idea to see what the default parameters are. We can use a the property `parameters` to easily see those default values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('learning_rate', [0.0001]),\n",
       "             ('epochs', [15]),\n",
       "             ('batch_size', [16]),\n",
       "             ('im_size', [299]),\n",
       "             ('architecture',\n",
       "              [<Architecture.resnet18: functools.partial(<function resnet18 at 0x7fb5b0f511e0>)>]),\n",
       "             ('transform', [True]),\n",
       "             ('dropout', [0.5]),\n",
       "             ('weight_decay', [0.01]),\n",
       "             ('training_schedule',\n",
       "              [<TrainingSchedule.head_first_then_body: 'head_first_then_body'>]),\n",
       "             ('discriminative_lr', [False]),\n",
       "             ('one_cycle_policy', [True])])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sweeper.parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we know the defaults, we can pass it the parameters we want to test. \n",
    "\n",
    "In this notebook, we want to see the effect of different learning rates across different image sizes using only 10 epochs (the default number of epochs is 15). To do so, I would run the `update_parameters` functions as follows:\n",
    "\n",
    "```python\n",
    "sweeper.update_parameters(learning_rate=[1e-3, 1e-4, 1e-5], im_size=[299, 499], epochs=[10])\n",
    "```\n",
    "\n",
    "Notice that all parameters must be passed in as a list, including single values such the number of epochs.\n",
    "\n",
    "These parameters will be used to calculate the number of permutations to run. In this case, we've pass in three options for learning rates, two for image sizes, and one for number of epochs. This will result in 3 X 2 X 1 total permutations (in otherwords, 6 permutations). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<utils_cv.classification.parameter_sweeper.ParameterSweeper at 0x7fb5f3da2a20>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sweeper.update_parameters(learning_rate=LEARNING_RATES, im_size=IM_SIZES, epochs=EPOCHS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our parameters defined, we call the `run()` function with the dataset to test on. We can also optionally pass in:\n",
    "- the number of repetitions to run each permutation (default is 3)\n",
    "- whether or not we want the training to stop early if the metric (accuracy) doesn't improve by 0.01 (1%) over 3 epochs (default is False)\n",
    "\n",
    "The `run` function returns a multi-index dataframe which we can work with right away."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "this Learner object self-destroyed - it still exists, but no longer usable\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>duration</th>\n",
       "      <th>accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"12\" valign=\"top\">0</th>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.0001]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.082297</td>\n",
       "      <td>0.840909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.064061</td>\n",
       "      <td>0.772727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.0001]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.141352</td>\n",
       "      <td>0.818182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.463922</td>\n",
       "      <td>0.636364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.001]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>20.722259</td>\n",
       "      <td>0.909091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.416063</td>\n",
       "      <td>0.886364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.001]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>24.365971</td>\n",
       "      <td>0.863636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.313704</td>\n",
       "      <td>0.886364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.133374</td>\n",
       "      <td>0.318182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.168197</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.146123</td>\n",
       "      <td>0.363636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.318881</td>\n",
       "      <td>0.409091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"12\" valign=\"top\">1</th>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.0001]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.138748</td>\n",
       "      <td>0.795455</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.129084</td>\n",
       "      <td>0.772727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.0001]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.593981</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.236939</td>\n",
       "      <td>0.727273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.001]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.004125</td>\n",
       "      <td>0.886364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.080764</td>\n",
       "      <td>0.931818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.001]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.212029</td>\n",
       "      <td>0.818182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.104657</td>\n",
       "      <td>0.909091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.112240</td>\n",
       "      <td>0.272727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.282302</td>\n",
       "      <td>0.340909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.820476</td>\n",
       "      <td>0.363636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.316323</td>\n",
       "      <td>0.136364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"12\" valign=\"top\">2</th>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.0001]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.363203</td>\n",
       "      <td>0.818182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.172707</td>\n",
       "      <td>0.681818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.0001]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.387194</td>\n",
       "      <td>0.772727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.345343</td>\n",
       "      <td>0.727273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.001]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.108574</td>\n",
       "      <td>0.931818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>15.990458</td>\n",
       "      <td>0.840909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 0.001]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.637141</td>\n",
       "      <td>0.863636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.611178</td>\n",
       "      <td>0.863636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|[batch_size: 16]|[im_size: 299]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>16.262137</td>\n",
       "      <td>0.318182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>16.624743</td>\n",
       "      <td>0.363636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|[batch_size: 16]|[im_size: 499]|[arch: resnet18]|[transforms: True]|[dropout: 0.5]|[weight_decay: 0.01]|[training_schedule: head_first_then_body]|[discriminative_lr: False]|[one_cycle_policy: True]</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <td>23.153472</td>\n",
       "      <td>0.227273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <td>23.406054</td>\n",
       "      <td>0.204545</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                              duration  \\\n",
       "0 PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           16.082297   \n",
       "                                                     fridgeObjectsWatermark  16.064061   \n",
       "  PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           23.141352   \n",
       "                                                     fridgeObjectsWatermark  23.463922   \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           20.722259   \n",
       "                                                     fridgeObjectsWatermark  16.416063   \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           24.365971   \n",
       "                                                     fridgeObjectsWatermark  23.313704   \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           16.133374   \n",
       "                                                     fridgeObjectsWatermark  16.168197   \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           23.146123   \n",
       "                                                     fridgeObjectsWatermark  23.318881   \n",
       "1 PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           16.138748   \n",
       "                                                     fridgeObjectsWatermark  16.129084   \n",
       "  PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           23.593981   \n",
       "                                                     fridgeObjectsWatermark  23.236939   \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           16.004125   \n",
       "                                                     fridgeObjectsWatermark  16.080764   \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           23.212029   \n",
       "                                                     fridgeObjectsWatermark  23.104657   \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           16.112240   \n",
       "                                                     fridgeObjectsWatermark  16.282302   \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           23.820476   \n",
       "                                                     fridgeObjectsWatermark  23.316323   \n",
       "2 PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           16.363203   \n",
       "                                                     fridgeObjectsWatermark  16.172707   \n",
       "  PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           23.387194   \n",
       "                                                     fridgeObjectsWatermark  23.345343   \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           16.108574   \n",
       "                                                     fridgeObjectsWatermark  15.990458   \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           23.637141   \n",
       "                                                     fridgeObjectsWatermark  23.611178   \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           16.262137   \n",
       "                                                     fridgeObjectsWatermark  16.624743   \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           23.153472   \n",
       "                                                     fridgeObjectsWatermark  23.406054   \n",
       "\n",
       "                                                                             accuracy  \n",
       "0 PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           0.840909  \n",
       "                                                     fridgeObjectsWatermark  0.772727  \n",
       "  PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           0.818182  \n",
       "                                                     fridgeObjectsWatermark  0.636364  \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           0.909091  \n",
       "                                                     fridgeObjectsWatermark  0.886364  \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           0.863636  \n",
       "                                                     fridgeObjectsWatermark  0.886364  \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           0.318182  \n",
       "                                                     fridgeObjectsWatermark  0.500000  \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           0.363636  \n",
       "                                                     fridgeObjectsWatermark  0.409091  \n",
       "1 PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           0.795455  \n",
       "                                                     fridgeObjectsWatermark  0.772727  \n",
       "  PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           0.750000  \n",
       "                                                     fridgeObjectsWatermark  0.727273  \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           0.886364  \n",
       "                                                     fridgeObjectsWatermark  0.931818  \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           0.818182  \n",
       "                                                     fridgeObjectsWatermark  0.909091  \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           0.272727  \n",
       "                                                     fridgeObjectsWatermark  0.340909  \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           0.363636  \n",
       "                                                     fridgeObjectsWatermark  0.136364  \n",
       "2 PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           0.818182  \n",
       "                                                     fridgeObjectsWatermark  0.681818  \n",
       "  PARAMETERS [learning_rate: 0.0001]|[epochs: 10]... fridgeObjects           0.772727  \n",
       "                                                     fridgeObjectsWatermark  0.727273  \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           0.931818  \n",
       "                                                     fridgeObjectsWatermark  0.840909  \n",
       "  PARAMETERS [learning_rate: 0.001]|[epochs: 10]|... fridgeObjects           0.863636  \n",
       "                                                     fridgeObjectsWatermark  0.863636  \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           0.318182  \n",
       "                                                     fridgeObjectsWatermark  0.363636  \n",
       "  PARAMETERS [learning_rate: 1e-05]|[epochs: 10]|... fridgeObjects           0.227273  \n",
       "                                                     fridgeObjectsWatermark  0.204545  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = sweeper.run(datasets=DATA, reps=REPS); df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the CLI <a name=\"cli\"></a>\n",
    "\n",
    "Instead of using python to run this experiment, we may want to test from the CLI. We can do so by using the `scripts/sweep.py` file. \n",
    "\n",
    "To reproduce the same test (different learning rates across different image sizes using only 10 epochs), and the same settings (3 repetitions, and no early_stopping) we can run the following:\n",
    "\n",
    "```sh\n",
    "python scripts/sweep.py \n",
    "    --learning-rates 1e-3 1e-4 1e-5\n",
    "    --im-size 99 299\n",
    "    --epochs 10 \n",
    "    --repeat 3 \n",
    "    --no-early-stopping \n",
    "    --inputs <my-data-dir>\n",
    "    --output lr_bs_test.csv\n",
    "```\n",
    "\n",
    "Additionally, we've added an output parameter, which will automatically dump our dataframe into a csv file. To simplify the command, we can use the acryonyms of the params. We can also remove `--no-early-stopping` as that is the default behavior.\n",
    "\n",
    "```sh\n",
    "python scripts/sweep.py -lr 1e-3 1e-4 1e-5 -is 99 299 -e 10 -i <my-data-dir> -o lr_bs_test.csv\n",
    "```\n",
    "\n",
    "Once the script completes, load the csv into a dataframe to explore it's contents. We'll want to specify `index_col=[0, 1, 2]` since it is a multi-index dataframe.\n",
    "\n",
    "```python\n",
    "df = pd.read_csv(\"data/lr_bs_test.csv\", index_col=[0, 1, 2])\n",
    "```\n",
    "\n",
    "HINT: You can learn more about how to use the script with the `--help` flag.\n",
    "\n",
    "```python\n",
    "python scripts/sweep.py --help\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize Results <a name=\"visualize\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we read in multi-index dataframe, index 0 represents the run number, index 1 represents a single permutation of parameters, and index 2 represents the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see the results, show the df using the `clean_sweeper_df` helper function. This will display all the hyperparameters in a nice, readable way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = clean_sweeper_df(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we've run our benchmarking over 3 repetitions, we may want to just look at the averages across the different __run numbers__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">P: [learning_rate: 0.0001] [im_size: 299]</th>\n",
       "      <th colspan=\"2\" halign=\"left\">P: [learning_rate: 0.0001] [im_size: 499]</th>\n",
       "      <th colspan=\"2\" halign=\"left\">P: [learning_rate: 0.001] [im_size: 299]</th>\n",
       "      <th colspan=\"2\" halign=\"left\">P: [learning_rate: 0.001] [im_size: 499]</th>\n",
       "      <th colspan=\"2\" halign=\"left\">P: [learning_rate: 1e-05] [im_size: 299]</th>\n",
       "      <th colspan=\"2\" halign=\"left\">P: [learning_rate: 1e-05] [im_size: 499]</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "      <th>fridgeObjects</th>\n",
       "      <th>fridgeObjectsWatermark</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>duration</th>\n",
       "      <td>16.194749</td>\n",
       "      <td>16.121951</td>\n",
       "      <td>23.374176</td>\n",
       "      <td>23.348735</td>\n",
       "      <td>17.611653</td>\n",
       "      <td>16.162428</td>\n",
       "      <td>23.738380</td>\n",
       "      <td>23.343180</td>\n",
       "      <td>16.16925</td>\n",
       "      <td>16.358414</td>\n",
       "      <td>23.373357</td>\n",
       "      <td>23.347086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>accuracy</th>\n",
       "      <td>0.818182</td>\n",
       "      <td>0.742424</td>\n",
       "      <td>0.780303</td>\n",
       "      <td>0.696970</td>\n",
       "      <td>0.909091</td>\n",
       "      <td>0.886364</td>\n",
       "      <td>0.848485</td>\n",
       "      <td>0.886364</td>\n",
       "      <td>0.30303</td>\n",
       "      <td>0.401515</td>\n",
       "      <td>0.318182</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         P: [learning_rate: 0.0001] [im_size: 299]                         \\\n",
       "                                     fridgeObjects fridgeObjectsWatermark   \n",
       "duration                                 16.194749              16.121951   \n",
       "accuracy                                  0.818182               0.742424   \n",
       "\n",
       "         P: [learning_rate: 0.0001] [im_size: 499]                         \\\n",
       "                                     fridgeObjects fridgeObjectsWatermark   \n",
       "duration                                 23.374176              23.348735   \n",
       "accuracy                                  0.780303               0.696970   \n",
       "\n",
       "         P: [learning_rate: 0.001] [im_size: 299]                         \\\n",
       "                                    fridgeObjects fridgeObjectsWatermark   \n",
       "duration                                17.611653              16.162428   \n",
       "accuracy                                 0.909091               0.886364   \n",
       "\n",
       "         P: [learning_rate: 0.001] [im_size: 499]                         \\\n",
       "                                    fridgeObjects fridgeObjectsWatermark   \n",
       "duration                                23.738380              23.343180   \n",
       "accuracy                                 0.848485               0.886364   \n",
       "\n",
       "         P: [learning_rate: 1e-05] [im_size: 299]                         \\\n",
       "                                    fridgeObjects fridgeObjectsWatermark   \n",
       "duration                                 16.16925              16.358414   \n",
       "accuracy                                  0.30303               0.401515   \n",
       "\n",
       "         P: [learning_rate: 1e-05] [im_size: 499]                         \n",
       "                                    fridgeObjects fridgeObjectsWatermark  \n",
       "duration                                23.373357              23.347086  \n",
       "accuracy                                 0.318182               0.250000  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.mean(level=(1,2)).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the average accuracy over the different runs for each dataset independently."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Text(0, 0, '0.00%'),\n",
       " Text(0, 0, '20.00%'),\n",
       " Text(0, 0, '40.00%'),\n",
       " Text(0, 0, '60.00%'),\n",
       " Text(0, 0, '80.00%'),\n",
       " Text(0, 0, '100.00%')]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtgAAAIkCAYAAAAkg1xnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzde7xWdZn//9clYKACmuLkGSobRBKELYYHLC0PYyId7KvjqGWJpmk6jWZWamoOhjNT+vMQpZONlKZZmmZOpWaGJ44aoonmAXUMj3kO5Pr9cd/sNrjZ+96w2Ote7Nfz8diPfa91r3Wv6+Zyu9973Z/1WZGZSJIkSSrGWmUXIEmSJK1JDNiSJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgQzYkiRJUoF6l11A0TbaaKMcPHhw2WVIkiRpDTdjxoxnM3PQ8uvXuIA9ePBgpk+fXnYZkiRJWsNFxGPtrXeIiCRJklQgA7YkSZJUIAO2JEmSVKA1bgy2JElacy1atIgFCxbwxhtvlF2KepC+ffuy+eab06dPn4a2N2BLkqTKWLBgAf3792fw4MFERNnlqAfITJ577jkWLFjAkCFDGtrHISKSJKky3njjDTbccEPDtbpNRLDhhht26VMTA7YkSaoUw7W6W1f/mzNgS5IkdcF5553HNttsw8EHH7zM+unTp3Pccce1u8/gwYN59tlnV/qYU6ZMYejQoQwdOpQxY8Zw++23d/raF198MT/84Q+7fKwXX3yRCy+8cKVrlWOwJUlShQ0++YZCX+/RSft2us2FF17IjTfeuMx43MWLF9PS0kJLS0uh9QBcf/31fPe73+X2229no402YubMmUyYMIG7776bd73rXSvc76ijjlqp4y0N2EcfffTKltzjeQZbkiSpQUcddRSPPPII48ePZ+DAgUycOJE999yTQw89lFtvvZWPfvSjADz33HPsueeebL/99hx55JFkZutrnHnmmQwdOpSPfOQjHHTQQZx77rkAPPzww+y9996MHj2aXXfdlQceeACAc845h8mTJ7PRRhsBMGrUKA477DAuuOCC1tecPHkyY8aMYcyYMcyfPx+A008/vdPXfuaZZ/jYxz7GiBEjGDFiBNOmTePkk0/m4YcfZuTIkZx44ok8/fTTjBs3jpEjRzJ8+HB+//vfr+Z/5eozYEuSJDXo4osvZtNNN+WWW27hhBNOYMaMGVx77bX86Ec/Wma7b3zjG+yyyy7MmjWL8ePH8/jjjwO1YSQ//elPmTVrFtdccw3Tp09v3WfixImcf/75zJgxg3PPPbf1DPLcuXMZPXr0Mq/f0tLC3LlzW5cHDBjA3XffzRe+8AWOP/74t9W9otc+7rjj2G233ZgzZw4zZ85k2223ZdKkSbznPe9h9uzZTJ48mR/96EfstddezJ49mzlz5jBy5Mhi/jHXYA4RkSRJWknjx4+nX79+b1t/2223cc011wCw7777ssEGGwBw++23s//++7fus99++wHwyiuvMG3aNA444IDW13jzzTdXeNzMXObCu4MOOqj1+wknnLDMth299s0339w6TrtXr14MHDiQF154YZn9d9hhBw4//HAWLVrEhAkTDNgNMGBLkiStpHXXXXeFz7U380TboSJtLVmyhPXXX5/Zs2e/7blhw4YxY8YMdt9999Z1M2fOZNiwYe0ea/njdvTajRg3bhy33XYbN9xwA4cccggnnngihx566Eq9Vk9hwJZUSUVf2NSZRi58kqSlxo0bx9SpU/na177GjTfe2HpWeJddduHII4/kK1/5CosXL+aGG27giCOOYMCAAQwZMoSrrrqKAw44gMzk3nvvZcSIEZx00kl8+ctf5le/+hUbbrghs2fP5gc/+AF33XVX6/GuvPJKTj75ZK688krGjh27TC0dvfYee+zBRRddxPHHH89bb73Fq6++Sv/+/Xn55Zdb93/sscfYbLPNOOKII3j11VeZOXOmAbsTBmxJkqSCnXbaaRx00EGMGjWK3XbbjS233BKoDbcYP348I0aMYKuttqKlpYWBAwcCMHXqVD7/+c9z1llnsWjRIg488EBGjBjB+PHjefLJJ9lpp52ICPr378/ll1/OJpts0nq8N998kx133JElS5bw4x//uHX90rPZK3rt73znO0ycOJFLLrmEXr16cdFFFzF27Fh23nlnhg8fzj777MPw4cOZPHkyffr0Yb311lupqf96mljRRxVV1dLSkm0vGJC0ZvIMttQzzZs3j2222absMlbJK6+8wnrrrcdrr73GuHHjmDJlCqNGjSr8OMceeyyjRo3iM5/5TOGv3RO1999eRMzIzLfNzegsIpIkSd1o4sSJjBw5klGjRvGJT3xitYTrr3/969x1112MHz++8NdW5xwiIkmS1I2Wn9JvdTjzzDM588wzV/tx1D7PYEuSJEkFMmBLkiRJBTJgS5IkSQVyDLYkNeL0gd18vJe693hruu7sn72TejzPYEuSJHXBeeedxzbbbMPBBx+8zPrp06dz3HHHtbvP4MGDefbZZ1f6mFOmTGHo0KEMHTqUMWPGcPvtt3f62hdffPFKzVn94osvcuGFF3a4zXe+8x2OP/741uUjjzySD3/4w63L559//gr/LZY6++yzu1zb6rDeeusV/pqewZYkSdVV9KcTDXwCceGFF3LjjTcyZMiQ1nWLFy+mpaWFlpa3TYm8yq6//nq++93vcvvtt7PRRhsxc+ZMJkyYwN1338273vWuFe531FFHrdTxlgbso48+eoXb7LTTTkydOrV1efbs2SxZsoS33nqLXr16MW3aNCZMmNDhcc4++2xOOeWULtW29PWLkJkrvHX9qvIMtiRJUoOOOuooHnnkEcaPH8/AgQOZOHEie+65J4ceeii33norH/3oRwF47rnn2HPPPdl+++058sgjlwlyZ555JkOHDuUjH/kIBx10EOeeey4ADz/8MHvvvTejR49m11135YEHHgDgnHPOYfLkyWy00UYAjBo1isMOO4wLLrig9TUnT57MmDFjGDNmDPPnzwfg9NNP7/S1n3nmGT72sY8xYsQIRowYwbRp0zj55JN5+OGHGTlyJCeeeCJPP/0048aNY+TIkQwfPpzf//73bL/99vzpT3/i9ddf56WXXmKdddZh5MiR3HfffQBMmzaNnXbaCYAJEyYwevRott12W6ZMmQLAySefzOuvv87IkSNbPwm4/PLLGTNmDCNHjuTII4/krbfeAmpnmE899VR23HFH7rjjDgYPHswpp5zC2LFjaWlpYebMmey111685z3v4eKLLwZqN/PZY489GDVqFO9///u59tprAXj00UfZZpttOProoxk1ahRPPPFE67/hs88+y9ixY7nhhlW/kZkBW5IkqUEXX3wxm266KbfccgsnnHACM2bM4Nprr33b3Nbf+MY32GWXXZg1axbjx4/n8ccfB2rDSH76058ya9YsrrnmGtrefXrixImcf/75zJgxg3PPPbf1DPLcuXMZPXr0Mq/f0tLC3LlzW5cHDBjA3XffzRe+8IVlhm509trHHXccu+22G3PmzGHmzJlsu+22TJo0ife85z3Mnj2byZMn86Mf/Yi99tqL2bNnM2fOHEaOHEnv3r0ZOXIk99xzD3feeSc77rgjH/jAB5g2bRpPPfUUmckWW2wBwKWXXsqMGTOYPn065513Hs899xyTJk2iX79+zJ49m6lTpzJv3jyuvPJK/vCHPzB79mx69erVeob81VdfZfjw4dx1113ssssuAGyxxRbccccd7Lrrrnz605/m6quv5s477+TUU08FoG/fvvzsZz9j5syZ3HLLLXzpS19q/SPnwQcf5NBDD2XWrFlstdVWQO0PjX333ZczzjiDffdd9Tv3OkREkiRpJY0fP55+/fq9bf1tt93GNddcA8C+++7LBhtsAMDtt9/O/vvv37rPfvvtB9TOuE6bNo0DDjig9TXefPPNFR43M4mI1uWDDjqo9fsJJ5ywzLYdvfbNN9/cOk67V69eDBw4kBdeeGGZ/XfYYQcOP/xwFi1axIQJExg5ciQAO++8M9OmTeP1119n7NixbL311px99tkMGjSo9ew11Mas/+xnPwPgiSee4KGHHmLDDTdc5hi//e1vmTFjBjvssAMAr7/+OhtvvHFrXZ/4xCeW2X7pHSrf//7388orr9C/f3/69+9P3759efHFF1l33XU55ZRTuO2221hrrbV48skneeaZZwDYaqut+MAHPtD6WosWLWKPPfbgggsuYLfddlvhv3lXGLAlSZJW0rrrrrvC59oG4KVWNOZ3yZIlrL/++syePfttzw0bNowZM2aw++67t66bOXMmw4YNa/dYyx+3o9duxLhx47jtttu44YYbOOSQQzjxxBM59NBD2Wmnnfjud7/LG2+8wTHHHMOgQYO4//77GTRoEDvvvDMAt956K7/5zW+44447WGeddfjgBz/IG2+88bZjZCaHHXYY//7v//625/r27fu2cdfveMc7AFhrrbVaHy9dXrx4MVOnTmXhwoXMmDGDPn36MHjw4NbjLt+z3r17M3r0aG666abCArZDRCRJkgo2bty41iEON954Y+tZ4V122YVf/OIXvPHGG7zyyiut430HDBjAkCFDuOqqq4Ba4JwzZw4AJ510El/+8pd57rnngNoFhT/4wQ+WuQjxyiuvbP0+duzYZWrp6LX32GMPLrroIqB2AeFf//pX+vfvz8svv9y6/2OPPcbGG2/MEUccwWc/+1lmzpwJ1C50vPPOO1m4cCEbb7wxEcGgQYO49tprW89gv/TSS2ywwQass846PPDAA9x5552tr9unTx8WLVrUWsfVV1/NX/7yFwCef/55HnvssZX95+ell15i4403pk+fPtxyyy0dvlZEcOmll/LAAw8wadKklT5mW57BliRJKthpp53GQQcdxKhRo9htt93Ycsstgdpwi/HjxzNixAi22morWlpaGDiwNhPK1KlT+fznP89ZZ53FokWLOPDAAxkxYgTjx4/nySefZKeddiIi6N+/P5dffjmbbLJJ6/HefPNNdtxxR5YsWcKPf/zj1vVLz2av6LW/853vMHHiRC655BJ69erFRRddxNixY9l5550ZPnw4++yzD8OHD2fy5Mn06dOH9dZbr3VIyQYbbMCgQYPYdtttW483duxY/vCHPzBixAgA9t57by6++GK22247/vEf/3GZoRkTJ05ku+22Y9SoUUydOpWzzjqLPffckyVLltCnTx8uuOCC1jHSXXXwwQez33770dLSwsiRIxk6dGiH2/fq1YsrrriC/fbbjwEDBnQ4g0ojYnVNT1KWlpaWbHvBgKQ10+CTV/0q7654tO8/d+vxvFlJwbzRzBpj3rx5bLPNNmWXsUpeeeUV1ltvPV577TXGjRvHlClTGDVqVOHHOfbYYxk1ahSf+cxnCn/tnqi9//YiYkZmvm1uRoeISJIkdaOJEycycuRIRo0axSc+8YnVEq6//vWvc9ddd7VeDKju5RARSZKkbrT8lH6rw5lnnsmZZ5652o+j9nkGW5IkSSqQZ7AlSd2u+8fQd+vhtJotPwe0tLp19ZpFz2BLkqTK6Nu3L88991yXA4+0sjKT5557jr59G/9L3TPYkiSpMjbffHMWLFjAwoULyy5FPUjfvn3ZfPPNG97egC1JkiqjT58+DBkypOwypA45RESSJEkqkAFbkiRJKpABW5IkSSqQAVuSJEkqkAFbkiRJKpABW5IkSSqQ0/Stom6/G9mkfbv1eJIkSeoaz2BLkiRJBTJgS5IkSQVqKGBHxAkRMTci/hgRP46IvhExJCLuioiHIuLKiFh7Bft+JSLmR8SDEbFXm/V719fNj4iT26yfGhH3RsTZbdZ9PSL2X5U3KkmSJHWHTgN2RGwGHAe0ZOZwoBdwIHAO8F+ZuTXwAvDZdvYdVt92W2Bv4MKI6BURvYALgH2AYcBBETEsIrYDyMztgF0jYmBEbAKMycxrV/3tSpIkSatXo0NEegP9IqI3sA7wNLA7cHX9+cuACe3stz9wRWa+mZl/BuYDY+pf8zPzkcz8G3BFfdtF9eOsBawNvAWcAZy6Mm9OkiRJ6m6dBuzMfBI4F3icWrB+CZgBvJiZi+ubLQA2a2f3zYAn2iwv3a7d9Zk5r36cmcBPgPcCkZmzOqoxIiZGxPSImL5w4cLO3pIkSZK02nQ6TV9EbEDt7PIQ4EXgKmpDO5aX7e2+gu3aC/YJkJnHtzn2L4AjI+KrwAjg15n5vbftmDkFmALQ0tLSXh2SJElSt2hkiMiHgT9n5sLMXARcA+wErF8fMgKwOfBUO/suALZos7x0uxWtb1W/qHE6sC4wPDM/BRwSEes0ULMkSZJUikYC9uPAByJinYgIYA/gfuAW4JP1bQ4D2rsI8TrgwIh4R0QMAbYG7gbuAbauz0SyNrULIa9bulNE9AG+CEymNuZ76VnppWOzJUmSpKbUyBjsu6hdzDgTuK++zxTgy8C/RsR8YEPgEoCIGB8RZ9T3nUttLPX9wK+AYzLzrfrY7S8ANwHzgJ/Ut13qGOCyzHwNuLf2snEf8IfMfHHV37YkSZK0ejR0q/TMPA04bbnVj1CbDWT5ba+jzdnozPwm8M12tvsl8MsVHO/bbR4ncFAjdUqSJEll806OkiRJUoEaOoMtrYkGn3xDtx7v0Un7duvxJElSOTyDLUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFah32QWoi04f2M3He6l7jydJklRxnsGWJEmSCmTAliRJkgpkwJYkSZIKZMCWJEmSCmTAliRJkgpkwJYkSZIKZMCWJEmSCmTAliRJkgpkwJYkSZIKZMCWJEmSCmTAliRJkgpkwJYkSZIKZMCWJEmSCmTAliRJkgpkwJYkSZIK1LvsAqQe4/SB3Xy8l7r3eJIkCfAMtiRJklQoA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCADtiRJklSghgJ2RKwfEVdHxAMRMS8ixkbEOyPi1xHxUP37BivY97D6Ng9FxGFt1o+OiPsiYn5EnBcRUV9/TkTcGxE/bLPtIRHxxVV9s5IkSdLq1ugZ7O8Av8rMocAIYB5wMvDbzNwa+G19eRkR8U7gNGBHYAxwWpsgfhEwEdi6/rV3RAwEdsrM7YBeEfH+iOgHfBq4cOXeoiRJktR9Og3YETEAGAdcApCZf8vMF4H9gcvqm10GTGhn972AX2fm85n5AvBrakF6E2BAZt6RmQn8sL7/EmDt+tnsfsAi4ETgvMxctArvU5IkSeoWjZzBfjewEPjviJgVEd+PiHWBf8jMpwHq3zduZ9/NgCfaLC+or9us/niZ9Zn5MvBTYBbwZ+AlYIfMvLZrb0uSJEkqRyMBuzcwCrgoM7cHXqWd4SArEO2syw7Wk5nfysyRmfkl4Ezg1Ij4XET8JCK+1u5BIiZGxPSImL5w4cIGS5MkSZKK10jAXgAsyMy76stXUwvcz9SHelD//pcV7LtFm+XNgafq6zdvZ32riNi+/vBPwKGZ+SlgeERsvfxBMnNKZrZkZsugQYMaeEuSJEnS6tFpwM7M/wOeiIh/rK/aA7gfuA5YOivIYUB7wzhuAvaMiA3qFzfuCdxUH1LyckR8oD7e+tB29j8TOBXoA/Sqr1sCrNPom5MkSZK6W+8GtzsWmBoRawOPAJ+hFs5/EhGfBR4HDgCIiBbgqMz8XGY+HxFnAvfUX+eMzHy+/vjzwA+oXcx4Y/2L+mtMAO7JzKfqy3dExH3AvZk5Z6XfrSRJkrSaNRSwM3M20NLOU3u0s+104HNtli8FLl3BdsNXcLyfAz9vs/xvwL81UqskSZJUJu/kKEmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFciALUmSJBXIgC1JkiQVyIAtSZIkFajhgB0RvSJiVkRcX18eEhF3RcRDEXFlRKy9gv2+EhHzI+LBiNirzfq96+vmR8TJbdZPjYh7I+LsNuu+HhH7r9xblCRJkrpPV85gfxGY12b5HOC/MnNr4AXgs8vvEBHDgAOBbYG9gQvrQb0XcAGwDzAMOCgihkXEdgCZuR2wa0QMjIhNgDGZeW3X354kSZLUvXo3slFEbA7sC3wT+NeICGB34J/rm1wGnA5ctNyu+wNXZOabwJ8jYj4wpv7c/Mx8pP76V9S3/TnQLyLWAtYG3gLOAE5dqXcnSZKq7fSB3Xy8l7r3eFojNXoG+9vAScCS+vKGwIuZubi+vADYrJ39NgOeaLO8dLt212fmPOBxYCbwE+C9QGTmrI6Ki4iJETE9IqYvXLiwwbckSZIkFa/TM9gR8VHgL5k5IyI+uHR1O5tme7uvYLv2gn0CZObxbY79C+DIiPgqMAL4dWZ+7207Zk4BpgC0tLS0V4ckSZLULRoZIrIzMD4i/gnoCwygdkZ7/YjoXT+LvTnwVDv7LgC2aLPcdrsVrQegflHjdGBdYHhmfioibouIqZn5WgN1S5IkSd2u0yEimfmVzNw8MwdTu2Dx5sw8GLgF+GR9s8OA9i5CvA44MCLeERFDgK2Bu4F7gK3rM5GsXX/d65buFBF9qF1UORlYh7+fHV86NluSJElqSqsyD/aXqV3wOJ/amOxLACJifEScAZCZc6mNpb4f+BVwTGa+VT/r/QXgJmozk/ykvu1SxwCX1c9U31t72bgP+ENmvrgKNUuSJEmrVUOziCyVmbcCt9YfP8LfZwRpu811tDkbnZnfpDb7yPLb/RL45QqO8+02jxM4qCt1SpIkSWXxTo6SJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgQzYkiRJUoEM2JIkSVKBDNiSJElSgXqXXYAkSZK6x+CTb+jW4z06ad9uPV6z8Ay2JEmSVCADtiRJklQgA7YkSZJUIAO2JEmSVCAvcpQkSQ3r9ovk+nbr4aRCeAZbkiRJKpABW5IkSSqQAVuSJEkqkAFbkiRJKpABW5IkSSqQAVuSJEkqkAFbkiRJKpABW5IkSSqQAVuSJEkqkAFbkiRJKpABW5IkSSqQAVuSJEkqkAFbkiRJKpABW5IkSSqQAVuSJEkqkAFbkiRJKpABW5IkSSpQ77ILkCRJ0hrq9IHdfLyXuvd4K+AZbEmSJKlABmxJkiSpQAZsSZIkqUAGbEmSJKlABmxJkiSpQAZsSZIkqUAGbEmSJKlABmxJkiSpQAZsSZIkqUAGbEmSJKlABmxJkiSpQAZsSZIkqUAGbEmSJKlAnQbsiNgiIm6JiHkRMTcivlhf/86I+HVEPFT/vsEK9j+svs1DEXFYm/WjI+K+iJgfEedFRNTXnxMR90bED9tse8jS40qSJEnNrJEz2IuBL2XmNsAHgGMiYhhwMvDbzNwa+G19eRkR8U7gNGBHYAxwWpsgfhEwEdi6/rV3RAwEdsrM7YBeEfH+iOgHfBq4cOXfpiRJktQ9Og3Ymfl0Zs6sP34ZmAdsBuwPXFbf7DJgQju77wX8OjOfz8wXgF9TC9KbAAMy847MTOCH9f2XAGvXz2b3AxYBJwLnZeaiVXifkiRJUrfo0hjsiBgMbA/cBfxDZj4NtRAObNzOLpsBT7RZXlBft1n98TLr6wH+p8As4M/AS8AOmXltJ3VNjIjpETF94cKFXXlLkiRJUqEaDtgRsR618Ht8Zv610d3aWZcdrCczv5WZIzPzS8CZwKkR8bmI+ElEfK29g2TmlMxsycyWQYMGNViaJEmSVLyGAnZE9KEWrqdm5jX11c/Uh3pQ//6XdnZdAGzRZnlz4Kn6+s3bWd/2mNvXH/4JODQzPwUMj4itG6lZkiRJKkMjs4gEcAkwLzP/s81T1wFLZwU5DGhvGMdNwJ4RsUH94sY9gZvqQ0pejogP1F//0Hb2PxM4FegD9KqvWwKs09A7kyRJkkrQyBnsnYFDgN0jYnb965+AScBHIuIh4CP1ZSKiJSK+D5CZz1MLyvfUv86orwP4PPB9YD7wMHDj0gNGxATgnsx8KjNfBO6IiPtqL5lzVvldS5IkSatJ7842yMzbaX/MNMAe7Ww/Hfhcm+VLgUtXsN3wFRzz58DP2yz/G/BvndUqSZIklc07OUqSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLktAvFW0AACAASURBVCRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBTJgS5IkSQUyYEuSJEkFMmBLkiRJBVqlgB0Re0fEgxExPyJObuf5d0TElfXn74qIwW2e+0p9/YMRsVd93aCIuD0i/hgRE9pse21EbLoqtUqSJEndYaUDdkT0Ai4A9gGGAQdFxLDlNvss8EJmvhf4L+Cc+r7DgAOBbYG9gQvrr3cQcBkwFjixvu1+wMzMfGpla5UkSZK6y6qcwR4DzM/MRzLzb8AVwP7LbbM/tcAMcDWwR0REff0VmflmZv4ZmF9/vUVAP+AdwJKI6A0cD0xehTolSZKkbrMqAXsz4Ik2ywvq69rdJjMXAy8BG3aw74+AvYBfAacDRwM/zMzXVqFOSZIkqdv0XoV9o5112eA27a7PzJeAfQEiYgPgy8DHI+J7wAbAf2TmHW8rJGIiMLG++EpEPNjYW6iegI2AZ7vtgN9or1VaGfau2uxftXVr/+xdofzZq7Ye0L+t2lu5KgF7AbBFm+XNgeXHSS/dZkF9uMdA4PkG9z0V+Ca1cdkzqJ3dvhb40PKFZOYUYMrKvpEqiYjpmdlSdh3qOntXbfav2uxfddm7auup/VuVISL3AFtHxJCIWJvaRYvXLbfNdcBh9cefBG7OzKyvP7A+y8gQYGvg7qU7RcTWwKaZ+TtgHWAJtTPffVehXkmSJGm1W+kz2Jm5OCK+ANwE9AIuzcy5EXEGMD0zrwMuAf4nIuZTO3N9YH3fuRHxE+B+YDFwTGa+1eblvwl8tf74x8DPgS9SO6stSZIkNa2onVBWVUTExPqQGFWMvas2+1dt9q+67F219dT+GbAlSZKkAnmrdEmSJKlABmxJkiSpQKsyTZ9Wo4j4eAObvZGZv1ztxajLImJUA5stysz7Vnsx6pKIeGcDmy3JzBdXezGSVBHmlmU5BrtJRcRz1Ob97mjG9HGZ+Z5uKkldEBEvU5vKsqP+DcnMwd1TkRoVEW9Qm5e/o971yswtu6kkdUFE/LWzTYCnM/N93VGPuiYi/rWBzV7NzO+u9mLUJeaWZXkGu3ndmJmHd7RBRFzeXcWoy+7JzN072iAibu6uYtQl8zJz+442iIhZ3VWMuuxh+1dpJwIX0XFIOwowYDcfc0sbnsGWpDYiom9mvrGq26gcEfHuzHxkVbdROSLiW5l50qpuI5XNgN3EImI9YG9qt5VfDDwE/G9mLim1MDUsIlpo07/MfKDkktSgiBgEbE6td3/OzFdKLkldFBEbAIsz8+Wya5F6AnPL3xmwm1REfIraR2VzgA8B06jN+vJ+4GAvjmtuEbEb8B/Ai8Bo4A/ABsAi4JDMfKLE8tSBiBgGnAcMBrYEZgEbA78DvpiZL5VXnToTEZsCk4D9gfWAJ+tPXQp8MzMXlVWbGhMR7wE+xrIh7cf+7DU3c8uynKaveX0N2C0zPwfsCGycmQcD/4Jjz6rg28A+mflhYBS1GUN2Br4JXFJqZerMpcAxmfleYBfggcwcQu2PJHvX/C4HLs3MgcABwE+Bbahdc3RBmYWpcxHxReBioC+wA9CPWtC+IyI+WGJp6py5pQ0DdvMK4PX641epnUEjM+8FBpRVlBrWKzMX1h8/DmwFkJm/BjYrrSo1ol9mPgiQmXdTO/tCZn4PGFZmYWrIhpl5K0BmXkNt1oJXM/NrwLhSK1MjPgfsnZlnAR8GhmXmV6kNO/ivUitTZ8wtbTiLSPP6JfCriPgdsA9wFbTO0dvR1dVqDtMj4hLgt9Q+qr4VICLWAXqVWJc693BEfJ1a7z4OzAaIiD74/8wqWBgR/wLcDHwCeBQgIgJPKlVFb+At4B1Af4DMfLz+M6jmZW5pwzHYTSwi/onaGbM59TOfRMRaQJ/MfLPU4tSh+i+CI6j3j9pH1m9FRD9qH5s9VmqBWqGIWB84hb/3blJmvhwRA4FtMvPOUgtUhyJiS+Bcav2bDZyYmU9HxIbABzPzp6UWqA7Vh4h8FriT2icO52Tmf9cvOv5pZvopRBMzt/ydAbsCvBJektRTRMS21MbN/9GZl6rJ3OLHZU0rIjaNiB9GxEvAs8DciHg8Ik73Y7LmFxHrRcQZEfHHiHgpIhZGxJ0R8emya1PHImKtiPhMRFwfEXMiYkZEXOEFVtURER+KiP8vIq6NiJ9GxKSIeG/ZdakxmTmX2qw9/SLi/fWp39TkzC3LMmA3L6+Er7apwCPULsz5BrVp3w4BPhQRZ5dZmDp1CbWLUicBtwA31Nd9LSKOLbMwdS4iJgGHUhtisIjaz+HDwFURcUCZtalzETEsIn4D3AHcBXwfuC8iflAfpqXmZW5pwyEiTSoi5mTmiDbLMzJzdP3xA5k5tLzq1Jl2+ndPZu5QH4t2v/1rXhFxb2Zu12b5zsz8QES8A5idmduUWJ46ERH3Zeb76497A7/LzJ3rH1n/PjOHl1uhOhIRdwKHZeaDETGG2pSZh0XEEcBemfnJkkvUCphbluUZ7Oa1MCL+pf6Ry7F4JXzVvBoRuwBExH7A8wD1u1n1uKupK2ZR/UYXRMQo4G8A9Qt0PCPR/JbUZy0A2JT6rD2Z+QL+7FWB02RWl7mlDaecal6HU7sS/mRqV8J/ob7+ncBXyipKDTsK+H5EvA/4I7V+Lr39do/7qKxiTgRuiYg3gD7AgdDau+vLLEwNORuYFREPAkOBz0Nr/+aUWZga4jSZ1WVuacMhIpK0nPoZlw0z89mya1HX1c9gvxuYn5kvll2PGuc0mVpTGLCbWETsBWwO/KbtvMkRcXhmXlpeZVoZEXFzZu5edh3qXEQMAAZl5sPLrd+uflcyVUhEnJ2Zp5Rdh9TTRMSfMvN9ZddRBgN2k6rPNLELMBPYD/h2Zp5ff25mZo4qsz51LCKWD2EBvA9YOrZwu7ftpKYQEZ8Cvg38hdoQkU9n5j315/zZa3IRcd7yq6jN4PNDgMw8rtuLUsMiohe126VvDtyYmdPaPPe1+i3U1YQi4mVq16m0vdZhHeA1IDOzR90uvccNOq+Q/YDdM/N4YDSwT0T8V/05L9Rpfo8C9wKfotbL/agFtqWP1bxOAUZn5kjgM8D/RMTH68/5s9f8Pk5tzOd0YEb9+6L64xkl1qXGfBfYDXgOOD8i/rPNcx9vfxc1iR8APwfem5n9M7M/8Hj9cY8K12DAbma9M3MxQH0M4X7AgIi4Cli71MrUqcwcT20O0CnAiMx8FFiUmY95m/Sm1yszn4bWWQw+BHw1Io7DWUSqYBtqN7nYm9rwusuAlzPzsvpjNbcxmfnPmfltYEdgvYi4pj5Npn/gNrHMPBb4DvDjiDiuPi1tj/1/pgG7eT0cEbstXcjMtzLzs9SGGDgPbwVk5s+AfYAPRsR1+IdRVby8dJo+gHrY/iCwP7BtWUWpMZn5cv2Tv/8ALo+If8PfdVXS+v/JzFycmROpzUhxM+AdHZtcZs4APlxf/B3Qt8RySuUY7CYVEf0AMvP1dp7bLDOf7P6qtLIiYgQwNjMvLrsWdazeq1czc/5y6/sAn8rMqeVUpq6qzwZzNLWfvX8pux51LiIuBy7PzF8tt/5zwEWZ2eNuuV1VEbEJsH1m/rLsWspgwJYkSZIK5MdmkiRJUoEM2JIkSVKBDNiSJElSgQzYFRAR13e0rOZm/6orIk7vaFnNzf5J5YiImR0t9wQG7Go4opNlNTf7V13L35jEG5VUi/2rMENadS1/x9ueeAdcZxGpgPqUfVtm5oNl16Kus3+SpJ4kIrYCts7M39R/B/bOzJfLrqs7eQa7yUXEftQm2f9VfXlk/aYlqgD7V10R8b6I+G1E/LG+vF1EfK3sutQY+1d9EbFVRHy4/rhfRPQvuyZ1LiKOAK6mdtt7gM2p3UK9RzFgN7/TgTHAiwCZORsYXGI96prTsX9V9T3gK8AigMy8Fziw1IrUFfavwgxplXYMsDPwV4DMfAjYuNSKSmDAbn6LM/OlsovQSrN/1bVOZt693LrFpVSilWH/qs2QVl1vZubfli5ERG+gx41HNmA3vz9GxD8DvSJi64g4H5hWdlFqmP2rrmcj4j3UfzFExCeBp8stSV1g/6rNkFZdv4uIU4B+EfER4CrgFyXX1O28yLHJRcQ6wFeBPeurbgLOysw3yqtKjbJ/1RUR7wamADsBLwB/Bv4lMx8tsy41xv5VW0R8i9rQukOBY4Gjgfsz86ulFqZORcRawGep/d4L4KbM/F65VXU/A3aTi4hdgWmZ+VabdaMy0+mKKsD+VV9ErAus1dOugF9T2L9qMqRVV0QcAvy87c9cRHw0M3vUPSAcItL8bgJujoh/aLPu+2UVoy6zfxUVEW9FxCTgtaW/KJyHtzrsX+UdDFyRmQdk5icz83sR8dGyi1JDzgd+HxHbtFl3RlnFlMWA3fweBCYDt0bETvV1UWI96hr7V11zqf0/8n8j4p31dfauOuxftRnSquvPwOHA1RFxQH1dj/vZ6112AepUZub1EfEgcGVEXIoXelSJ/auuxZl5UkR8itov+kOxd1Vi/6rtz9SGiFwdEadn5lX0wJBWUZmZMyNiN+DHEbEj0KvsorqbZ7CbX0DrFEW7AuOA7UqtSF1h/6prae9+AnwK+G/g3aVWpK6wf9WW9WtVdgMmRsS59MCQVlFPA2Tms8Be1P6wHV5qRSXwIscKiogtM/PxsuvQyrF/1RARozNzRpvlAcCEzPxhiWWpQfav2iLihszct/54LeAc4EuZ6YlBVYIBu0lFxEmZ+a2IOK+95zPzuO6uSY2zf9UVEbtn5s0R8fH2ns/Ma7q7JjXO/knliIhvZ+bxEfEL2hmOlZnjSyirNI7Bbl7z6t9ndLiVmpX9q67dgJuB/dp5LgEDWnOzfxVmSKu0/6l/P7fUKpqEZ7ArpP4x2XqZ+deya1HX2T9J6tjSoT31C+TeJjN/1901aeVFxAbAFpl5b9m1dDfHMjW5iPhRRAyo3yzhfuDBiDix7LrUGPtXXRHxxXrvIiK+HxEzI2LPzvdUM7B/1bR03Hxm/m7pF3Av8ILhuhoi4tb6z947gTnAf0fEf5ZdV3czYDe/YfUznhOAXwJbAoeUW5K6wP5V1+H13u0JbAx8BphUbknqAvtXYYa0ShtY/9n7OPDfmTka+HDJNXU7A3bz6xMRfagFtGszcxHO5Vol9q+6ls65+0/UfknMabNOzc/+VZshrbp6R8Qm1KbH7FG3R2/LgN38vgs8CqwL3BYRWwGO4a0O+1ddMyLif6kFtJsioj+wpOSa1Dj7V22GtOo6A7gJmJ+Z90TEu4GHSq6p23mRY8VERAC9MnNxffmwzLys5LLUIPtXHfWLUkcCj2TmixGxIbDZ0ot1ImLbzJxbapFaIftXbfVbbH8duD0zj66HtMmZ+YmSS9MqioivZOa/l13H6mbArriImJmZo8quQyvH/lWXvas2+1dtPSWkrYl6ys+eQ0SqzzGF1Wb/qsveVZv9q7YDyi5AK61H/OwZsKvPjyCqzf5Vl72rNvtXbT0ipK2hesTPngG7+vyfTLXZP0nquh4R0tZQPeL3ngG7+v5QdgFaJfavuv5WdgFaJfav2npESFtDXVV2Ad3BixybXESsDxwKDAZ6L12fmceVVZMaZ/+qLSK24+29u6a0gtQl9m/NFRGnZObZZdeht4uIIcCxvP1nb3xZNZWhd+ebqGS/BO4E7sM5XKvI/lVURFwKbAfM5e+9S8CAVgH2r9o6C2mG66b2c+AS4Bf04N97nsFucj1lOps1lf2rroi4PzOHlV2HVo79q7aImEMtpC1zciIzf1daUWpIRNyVmTuWXUfZDNhNLiJOAF6hdierN5euz8znSytKDbN/1RURlwD/kZn3l12Lus7+VZshrboi4p+BrYH/ZdnfezNLK6oEBuwmFxHHAN8EXuTvV01nZr67vKrUKPtXXRExjtpHnP9H7ZdEUOvddqUWpobYv2ozpFVXRPw7cAjwMG2GZ2Xm7uVV1f0M2E0uIh4GdszMZ8uuRV1n/6orIuYD/8rbP6J+rLSi1DD7V22GtOqKiAeA7TKzR8/U40WOzW8u8FrZRWil2b/qejwzryu7CK00+1dtHwPe3dNDWkXNAdYH/lJ2IWUyYDe/t4DZEXELy35M5jRv1WD/quuBiPgRtWEGbXvnLBTVYP+qzZBWXf9A7efvHpb92XOaPjWVn9e/VE32r7r6UfvlsGebdU7zVh32r9oMadV1WtkFNAPHYEuSpKYSEbu1t95p+lQVBuwmFRE/ycxPRcR9/H32iVZeCd/c7F91RcRJmfmtiDif9nvn8J4mZv+kckTE7Zm5S0S8zLI/e0tn8BlQUmmlcIhI8/pi/ftHS61CK8v+Vde8+vfppVahlWX/KsyQVl2ZuUv9e/+ya2kGnsGWJEmSCrRW2QWofRFxfRHbqBz2r7oi4vQitlE57J9Ujojo9CZAjWyzpvAMdpOKiBeB2zraBNjWOwI2J/tXXRGxAPjPjjYBjsjMod1UkrrA/lVbRMzMzFGruo26X0S8DjzU0SbAwMzcsptKKpVjsJvX/g1s4wT8zcv+Vdf3gM7GEH6vOwrRSrF/1bZNRNzbwfMBDOyuYtQljfzR+tZqr6JJeAZbkiQ1hYjYqoHN3srMBau9GGkVGLAlSZKkAnmRoyRJklQgx2BLkiRplUTEOxvYbElmvrjai2kCDhFpUhHx1842AZ7OzPd1Rz3qGvtXXRFxXQObPZ+Zn17dtajr7F+1GdKqKyLeAJ6i9vttRXo5i4jK9nBmbt/RBhExq7uKUZfZv+raBvhcB88HcEE31aKus3/V9hQNhDSgR4S0ipnn772/M2A3r08UtI3KYf+q66uZ+buONoiIb3RXMeoy+1dthrTqGlvQNmsEh4hIkqSmEBF9M/ONVd1GzSEi3pmZz5ddRxmcRaSCIuK+smvQyrN/1RURU8quQSvP/jW/9oLz8uOyDdfNKSK+1ubxsIj4EzAjIh6NiB1LLK0UDhFpUhHx8RU9BbyrO2tR19m/6urgIqsA/qk7a1HX2b9qi4ivZeZZ9cfDgJ8DfSIigP+XmXeVWqA68nHgrPrjycAXM/PGiBgDfBvYqbTKSmDAbl5XAlOB9sbw9O3mWtR19q+6FgKPsexFVllf3riUitQV9q/aDGlrhk0z80aAzLw7IvqVXVB3M2A3r3uBczPzj8s/EREfLqEedY39q65HgD0y8/Hln4iIJ0qoR11j/9YcPT6kVcy769NkBrB5RKyTma/Vn+tTYl2lMGA3r+OBFc2l/LHuLEQrxf5V17eBDYC3BTTgW91ci7rO/lWbIa269l9uuRdARPwDcFH3l1MuZxGRJElNISJ2W27VzMx8uR7SPpmZzmGuSjBgN6mI6A18ltrZzk2pjSF8CrgWuCQzF5VYnjph/6otIoZSOxuzGX/v3XWZOa/UwtQQ+yd1v4gYCHwFmAAMqq/+C7Xfe5N62t03naavef0PMBI4ndqV7/sC3wBGAJeXV5YaZP8qKiK+DFxB7SPqu4F76o9/HBEnl1mbOmf/qi0iBkbEpIh4ICKeq3/Nq69bv+z61KGfAC8AH8zMDTNzQ+BD9XVXlVpZCTyD3aQi4sHM/McVPPenzHxfd9ekxtm/6qrP3brt8p8yRMTawNzM3LqcytQI+1dtEXETcDNwWWb+X33du4DDgA9n5kfKrE8r1snvvRU+t6byDHbzeiEiDoiI1h5FxFoR8f+o/TWo5mb/qmsJtWE9y9uk/pyam/2rtsGZec7ScA2Qmf+XmecAW5ZYlzr3WEScVB8vD9QucKx/qtTjZvBxFpHmdSBwDnBhRLxA7SPO9an9ZX9gmYWpIfavuo4HfhsRD/H3XwpbAu8FvlBaVWqU/au2xyLiJGpnsJ+B1lkoPk0PDGkV8/+Ak4HfRcTSOeefAa4DPlVaVSVxiEgFRMSG1Hr1bNm1qOvsX/XUP3kYQ+0iuQAWAPdk5lulFqaG2L/qiogNqIW0/fn7jYGWhrRzMvP5smqTusIz2E1s+SvhI+Ip4NrMfKDcytQI+1dp2eZrSZvvqgb7V1GZ+QLw5fqXKi4irs/Mj5ZdRxkcg92kOrgS/gqvhG9+9q+6ImJP4CHePgPMQ/Xn1MTs35onIq4vuwattM3KLqAsDhFpUl4JX232r7oiYh6wT2Y+utz6IcAvM3ObUgpTQ+zfmiciZmXm9mXXoa6LiEsz8/Cy6yiDZ7Cbl1fCV5v9q67e1MbsLu9JvFVzFdi/Nc+ssgvQ/9/evcZaepZlHP9ftBkZgSLQCsFyqBQqBQo0qCAEEVGDHIwcwkGOFowplgrGLzZoIWgiUUJTQoBmaCsJqKjRIjE1JhQFEjmWCi0UbNoSoY0i0FrSlg63H9bazt7T2bPXmo77ee+1/r9kp7PeNSR3cuWZuVjz3u9aTpK9SU5Z13IN3oM9ZW7C92Z+fb0P+HSSP+dAdg9i9vSXfcOm0qLMb0Uk2Qs8eJ1LWkdJngP8CbAHOCnJ44C3VNVzx062u7xFZMLchO/N/PpK8kgOLKhuZHdJVV05dDAtxPz621zSqmptS1pHST4LPB24bOPWniRXVNVpYyfbXX6CPW1uwvdmfk1V1VXAVaPn0JExv5VwLrMPKC4DqKrLkzx03Dhawh1V9d0ko+cYyoI9UfNt93cx24b/j/nlE4GTk5xZVf84bDjtyPxWU5Jzq+rc0XPoyJhfK5a0vr6Y5KXAMUkeDrwe+OTgmXadBXu6zgOesd0mPOAm/LSZ32r67OgBdJeYXx+WtL7OAs4BbgM+AFwKvHXoRAN4D/ZEzZfjHllVdxx0fQ9wZVWdPGYyLcL8JOnIJflhZiVt49nllwJvrapbx00lLc5PsKfLTfjezK+pJMcCZwC/yuxRiwV8A/g7YN/BzzbXtJjfaqiq7zEr2OeMnkV3XZL3VtVvjJ5jN/kJ9oS5Cd+b+fWU5IPAd4CLOfA85ROBVwL3raoXjZpNOzO/1bWOJa2TJPfd7i3gC1V14m7OM5oFW5I2SfKVqjplm/eurqpH7PZMWpz59WZJ6yvJfuA6ZlltqPnrH6uqPUMGG8RvcmwoybmjZ9CRM7/J+3aSF86fYw7Mnmme5EXAtwfOpcWYX2//CXyG2ULqxs9n5j8/OnAu7ewa4GlVddKmnx+vqpOAG0cPt9u8B7snN+F7M79pezHwx8C7kmwUsh8BPjp/T9Nmfr1dA/x8VV1/8BtJvn6I36/peAdwH+BO2QFv2+VZhvMWEUnaRpL7Mftz8r9Gz6LlmV8/SV4HfLyqvnCI986qqvMHjCUtzYI9UW7C92Z+kiStLwv2RLkJ35v5SZK0vizYE+UmfG/mJ0nS+vIpItPlJnxv5idJ0pqyYE/Xi4EXADcmuTrJ1cANwPNwE74D81sBST53uNeaNvOTxvDseYtIC27C92Z+krS8JJ+rqtO3ey1NmQVbkraR5CHAw6vqn5LsBY6tqptHz6XFmJ80hmfPW0Qk6ZCSvBb4K+A980snAn87biItw/z6S/KQJM+Y/3pvknuNnkk78+zNWLAl6dBeBzwZuAmgqr6KX9Xcifk1ZklrzbOHBVuStnNbVd2+8WL+5UHeU9eH+fVmSevLs4cFuwW3cXszv7Y+luT3gL1JfgH4EPDhwTNpcebXmyWtL88eLjlK0iHNn2F+BvCLQIBLq+qCsVNpUebXW5K3Mfs23FcAZwFnAldW1TlDB9OOPHszFuwG3Mbtzfx6SnJ2VZ230zVNk/n1Zknry7M34y0iE+eiR2/m19orD3HtVbs9hI6Y+fV2VlVdUFUvrKoXVNUFSc4ePZQW4tkDjh09gHb0OuCngH+F2aJHEhc9+jC/ZpK8BHgpcFKSSza9dS/gW2Om0qLMb2W8Ejj4E89XHeKaJsKzt5UFe/puq6rbkwAuejRkfv18EvgmcDzwp5uu3wxcMWQiLcP8GrOktebZ28SCPX0Hb+OeyRpu4zZmfs1U1XXAdcCTRs+i5Zlfe5a0pjx7W7nkOHEuevRmfn0leSJwPvBIYA9wDHBLVR03dDAtxPykMTx7My45Tp+LHr2ZX1/vBF4CfBXYC7yG2V8a6sH8GkvyxCSfTvI/SW5Psj/JTaPn0kI8e1iwO3Abtzfza6yqvgYcU1X7q+pC4OdGz6TFmV9rlrTGPHvegz1ZLnr0Zn4r4XtJ9gCXz7/04pvAPQbPpMWZX3NV9bUkx1TVfuDCJJ8cPZMW4tnDgj1lLnr0Zn79vZzZv/L9FvAG4EHA84dOpGWYX2+WtL48e7jkKEl3kuQY4OKqetnoWbQ88+tv/g24NzJbknsDcG/gXfNbDzRRnr0DvAd74lz06M38epr/k/QJ80/Q1Iz59TYvaX9YVbdW1U1V9eaqeqPlevo8ewd4i8j0vRN4MfAh4AnAK4CTh06kZZhfX9cCn5jfQ3/LxsWqevuwibSMazG/lqpqf5ITkuypqttHz6OlXYtnz4LdgYsevZlfW9+Y/9yN2XKqejG/3q7FktaVZw8LdgcuevRmfk1V1ZsP936S86vqrN2aR8sxv/YsaU159mZccpw4Fz16M7/VleRzVXX66Dl0ZMyvt3UpaatoXc6en2BP2KZFj5cBtwKH/X+Fmhbzk6T/N08ePYB0OD5FZMLcxu3N/CRJWk9+gj191+KiR2fXYn6rKqMH0F1iftIYa3H2uKx5lgAAB/VJREFULNjT56JHb+a3us4bPYDuEvPrbS1K2opai7PnkmNzLnr0Zn7TleTDwMF/QH4X+Azwnqq6dfen0qLMb7UleVVVXTR6Dt2ZZ2/Ggt3cumzjrirzm64k5wEnAB+cX3oRcAOwFziuql4+ajbtzPx6s6T15dmb8RYRSTq0x1fVUze9/nCSf66qpyb50rCptCjz6+0a7lzSbgQeAVwArEVJa8qzhwVbkrZzQpIHV9X1AEkeDBw/f8+vb54+8+vNktaXZw8L9ipw0aM385uu3wE+nuTfmeV0EnBmknsAFw+dTIswv94saX159vAe7PZc9OjN/KYtyQ8BP8HsL4kve99nL+bXV5JfBt4NbClpwGXAa6vqHeOm0048exbsyXPRozfz6y3JzwAPZdO/9lXVnw0bSEsxv94saX159izYk+c2bm/m11eS9wMPAy4H9s8vV1W9ftxUWpT59WdJ68mzN2PBnriNpY5DXUvypap61KjZtDPz6yvJVcCp5R+SLZlfb5a0vjx7My45Tp+LHr2ZX19fBB4AfHP0IDoi5tfbE7CkdeXZw4Ldgdu4vZlfX8cDVyb5FHDbxsWqeu64kbQE8+vNktaXZw9vEWnBRY/ezK+nJD97qOtV9bHdnkXLM7/eknwUeByw1iWtI8/ejAW7ARc9ejM/SVqOJU3dWbAnzkWP3syvnyQfr6qnJLmZrY9YDLPsjhs0mhZgftIYnr2tLNgT5zZub+YnSYuzpGlVuOQ4fS569GZ+jSU5Brg/W2/vuX7cRFqG+fVTVU+Z//deo2fRkfPsWbA7cBu3N/NrKslZwB8ANwI/mF8u4LRhQ2lh5tefJa0nz96Mt4hMnIsevZlfX0m+Bvx0VX1r9Cxanvn1tl1Jq6q1KmkdefZm/AR74ixivZlfa18Hvjt6CB0x8+vtbOCUdS9pTXn2sGBPlosevZnfSrgGuCzJR9h6e8/bx42kJZhfb5a0vjx7WLAny0WP3sxvJVw//9kz/1Ev5tebJa0vzx4W7BZc9OjN/PqZZ3bPqvrd0bNoeea3EixpDXn2DrBgT5zbuL2ZX09VtT/J6aPn0JExv94saX159g6wYE+fix69mV9flye5BPgQcMvGxar6m3EjaQnm15QlrT3PHhbsDlz06M38+rov8C3g6ZuuFbBWf0k0Zn69WdL68uzhc7AnL8k+4BTARY+GzE+SlpfkwkNcrqr69V0fRjoCfoI9fS569GZ+TSW5O3AG8Cjg7hvX/Qu+B/PrrapePXoGHRnP3owFe8Jc9OjN/Np7P/Bl4JeAtwC/Blw1dCItw/was6S15tkD7jZ6AG2vqvYDLno0ZX7tnVxVbwJuqaqLgWcBjxk8kxZnfr29H3gAs5L2MeBE4OahE2lRnj38BLsDFz16M7++vj//73eSPBq4AXjouHG0JPPr7eSqemGSX6mqi5N8ALh09FBaiGcPC3YHbuP2Zn59vTfJfYA3AZcA9wR+f+xIWoL59WZJ68uzh08RkSRJE5PkNcBfM/tSrguZl7SqevfQwaQFWbAnzkWP3syvryT3B/4IeGBVPTPJqcCTqmrf4NG0APOTxvDszbjkOH0uevRmfn1dxOyezwfOX18N/PawabSsizC/tpLcP8m+JP8wf31qkjNGz6WFXIRnz4LdgNu4vZlfX8dX1V8CPwCoqjuA/WNH0hLMr7eLsKR15dnDgt3BwYse98ZFj07Mr69bktyP2VIqSZ6IX3vfifn1Zknry7OHTxHpwG3c3syvrzcyy+xhST4BnAC8YOxIWoL59WZJ68uzh0uOkrStJMcCpwABvlJV39/hf6IJMb++kpwOnA88Gvgi85JWVVcMHUwL8exZsCfPbdzezK+fJM873Pt+SdC0md/qsKT14tnbyoI9cfMN6guBc6rqsfM/cD5fVS7KNWB+/SS58DBvl49YnDbz682S1pdnbysL9sQl+XRV/WSSz1fV4+fXLq+qx42eTTszP0lanCVNq8Ilx+lz0aM382smybOr6u/v6u/RGObXW1W9evQMOjKeva38BHviXPTozfz6SXIV8FJm931u56KqOm2XRtISzK83S1pfnr2tLNgNuOjRm/n1kuQy5v/icBj/XVXP34VxtCTz682S1pdnbysL9kS56NGb+UnS8ixpWhXegz1dzznMewVY0KbN/CRpSVX1tNEzSEeDn2BLkiRJR9HdRg+gQ0vy7KPxezSG+UmStL78BHuiXPTozfxWQ5IHVNUN273WtJmfNIZnz4I9WS569GZ+qyHJR6rqWdu91rSZX2+WtL48exZsSZI0QZY0dWbBlqRNktwd+E3gZOAK4H1VdcfYqbQo85PG8OxtZcGWpE2S/AXwfeBfgGcC11XV2WOn0qLMrzdLWl+eva0s2JK0SZJ/q6rHzH99LPCpqjp98FhakPn1Zknry7O3lV8004CLHr2ZXzv/91X2VXVHcrgHwWiCzK+3UzeVtH3ApwbPo8V59jbxOdg97NvhtabN/Hp5bJKb5j83A6dt/DrJTaOH047Mr7ctJW3kIFqaZ28TbxGRJEmTkGQ/cMvGS2Av8L35r6uqjhs1m7QMC/ZEuejRm/lJkrS+LNgT5aJHb+YnSdL6smBPlNu4vZmfJEnryyXH6XLRozfzkyRpTfkJ9kS56NGb+UmStL4s2JIkSdJR5C0ikiRJ0lFkwZYkSZKOIgu2JEmSdBRZsCVJkqSjyIItSZIkHUUWbEmSJOko+l/zHdjJ4iybaAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax = df.mean(level=(1,2))[\"accuracy\"].unstack().plot(kind='bar', figsize=(12, 6))\n",
    "ax.set_yticklabels(['{:,.2%}'.format(x) for x in ax.get_yticks()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additionally, we may want simply to see which set of hyperparameters perform the best across the different __datasets__. We can do that by averaging the results of the different datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>P: [learning_rate: 0.0001] [im_size: 299]</th>\n",
       "      <th>P: [learning_rate: 0.0001] [im_size: 499]</th>\n",
       "      <th>P: [learning_rate: 0.001] [im_size: 299]</th>\n",
       "      <th>P: [learning_rate: 0.001] [im_size: 499]</th>\n",
       "      <th>P: [learning_rate: 1e-05] [im_size: 299]</th>\n",
       "      <th>P: [learning_rate: 1e-05] [im_size: 499]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>duration</th>\n",
       "      <td>16.158350</td>\n",
       "      <td>23.361455</td>\n",
       "      <td>16.887041</td>\n",
       "      <td>23.540780</td>\n",
       "      <td>16.263832</td>\n",
       "      <td>23.360222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>accuracy</th>\n",
       "      <td>0.780303</td>\n",
       "      <td>0.738637</td>\n",
       "      <td>0.897727</td>\n",
       "      <td>0.867424</td>\n",
       "      <td>0.352273</td>\n",
       "      <td>0.284091</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          P: [learning_rate: 0.0001] [im_size: 299]  \\\n",
       "duration                                  16.158350   \n",
       "accuracy                                   0.780303   \n",
       "\n",
       "          P: [learning_rate: 0.0001] [im_size: 499]  \\\n",
       "duration                                  23.361455   \n",
       "accuracy                                   0.738637   \n",
       "\n",
       "          P: [learning_rate: 0.001] [im_size: 299]  \\\n",
       "duration                                 16.887041   \n",
       "accuracy                                  0.897727   \n",
       "\n",
       "          P: [learning_rate: 0.001] [im_size: 499]  \\\n",
       "duration                                 23.540780   \n",
       "accuracy                                  0.867424   \n",
       "\n",
       "          P: [learning_rate: 1e-05] [im_size: 299]  \\\n",
       "duration                                 16.263832   \n",
       "accuracy                                  0.352273   \n",
       "\n",
       "          P: [learning_rate: 1e-05] [im_size: 499]  \n",
       "duration                                 23.360222  \n",
       "accuracy                                  0.284091  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.mean(level=(1)).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make it easier to see which permutation did the best, we can plot the results using the `plot_sweeper_df` helper function. This plot will help us easily see which parameters offer the highest accuracies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtAAAAKhCAYAAAB5KZn4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdeZyVdd3/8dcHQZQElYRCQTHFHNkGGxXScikEcRek0FstKevn7S6aZmiibbe5pd6akkG4m7lUKN1C7hEMMhKK5BLKlo6KKyjb9/fHOUzDMANc45w5M8Pr+XjMg3N9r+1zzXHG9/nO9/pekVJCkiRJ0sZpVewCJEmSpObEAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZuEiNgsIj6MiB0bctvmKiJOioiHC3TsLSLixYjoXIjjN6SI+HpEzNuI7c6JiMsboSRJzYABWlKTlA+wa75WR8SyasvHZz1eSmlVSmmrlNLrDbltVhFxeUSsiIgP8l9zI+JXEfH5hj5XtXPuGhFrTfqfUhqfUjqkQKf8f8CjKaU3C3T8YrgJ+HZEfLbYhUgqPgO0pCYpH2C3SiltBbwOHF6t7faa20dE68avst5uTym1Bz4LDAW6AeUR8bn6HCwiNmvI4hrA94AJxS6iIaWUlgJ/AU4odi2Sis8ALalZyvfk3h0Rd0bEB8B/RcSAiJgaEe9GxOJ8z26b/PatIyJFRPf88m359Q/ne4L/FhE7Z902v/6QiPhnRLwXEddFxNMR8a0NXUNKaXlKaTZwLPAucHb+eN+JiMeqHb+2em6IiEci4iPgKxFxRERU5Ot7PSJGVzvVE/n91vTg71XLOfaLiPL8NUyLiH2qrXsqIi6NiGfyx38kIjrW8b58gfwHgmpth0XEnPy+CyLi7GrrjoiI5/Lv2VMR0avaup0i4oGIqIyItyLi2nx7q4i4OCJei4g3I2JcRHTIr9s1/706MX+uyoi4oNox20XEhIhYEhHPA1+qUf8PI2JRRLwfuWEoB1Rb/RhwaG3XLWnTYoCW1JwdDdwBbA3cDawEzgS2A/YFBpPrDa3LccBooCO5Xu7Lsm4buXG+9wDn5c/7L2DvLBeRUloJPAR8JcNuxwGXAu2BvwEfAv9F7ntxOHBmRByW3/ar+fOs6cGfXv1AEbEd8GfgSnK94r8CJkbEtjXOdxLwOeAzwDl11NUbeDmltKpa22+Bkfle9z7A4/nz7gXcAnwnf95bgQcjYvP8XxT+DLwMdCcXyu/JH+87+Ws9ANgF2Ba4tkYdXwZ2BQYBl0ZEj3z7mPyxvgAMyV/Tmu9DT3L/veyZUuoAHELuvV5jDtC3juuWtAkxQEtqzp5KKf0xpbQ6pbQspTQ9pfT3lNLKlNKrwM3A/uvZ//cppfKU0grgdqC0HtseBlSklB7Mr7saeKse17KIXDjfWPenlP6Wv/ZPUkpTUkqz88vPAXex/muv7nDg+ZTSnfnv3W3Aq6zd2/qblNJL+aEM91L392ob4IMabSuAPSKifUrpnZTSs/n2U4D/zb9vq1JKt+bb9wIGkPtA8oOU0kf59/fp/PrjgV+mlP6VUvoA+CFwXERU/3/aj1NKH+fP9Tz/Cb7DgctTSktSSq8B11fbZyWwBdAzIlrnj/9qtfUf5K9P0ibOAC2pOZtffSEido+IP0fEvyPifXK9jdutZ/9/V3u9FNiqHttuX72OlFICFmxE7TXtALyTYfua1z4gIh7LD1l4j1wv7fquvbrtgddqtL2Wr2mNjf1eLSHXK17d0cARwOv5GtcMD9kJ+EF++Ma7EfEu0CV/3m7AvBo92XXV+xqwOdBpTUNKqa56u7D29+61avvMBc4l99/Nm/nhQdVv7mxPbqiNpE2cAVpSc5ZqLP8amA3smv8T/MVAFLiGxUDXNQsREawdPDcocjcBHg48mW/6CGhXbZPaZuioee13AfcB3VJKWwNj+c+119y2pkXkwmx1OwILN7BfbWYBu0S1GxvzfxU4AugM/ClfK+SC7KUppW2qfbVLKd2TX7dT1H6DZM16dwSWA5UbUd+/yYXz6vtWSSndllLaF9gZ2Az4WbXVJcBzG3EOSS2cAVpSS9IeeA/4KCJKWP/454byJ2DPiDg8P273TKr1hK5PRLSJiD3IBcqOwDX5Vc8BfSKid0RsCVyyEYdrD7yTUvo4IvoD36y27k0g5W/wq+saekbEN/I3LB5HbvzwxI25jupSSvPIjRv+EkBEbBkRx0VEh/wQlw+ANb3KNwP/nb+pMSJiq/z38TPkxnW/Dfw0f+PflhGxb36/O4FzIqJ7RLQHfgLcmVJavREl3gP8MCK2idw836etWRERJRFxYES0BZblv6r3gO8PFGTubEnNiwFaUktyLrmbwj4g1xt9d6FPmFJ6A/gGcBW5wLcLMBP4ZD27HR+5mUOWAA8CbwBla4YdpJReAH5KbtaHueRn0diA/wf8LH/cH/KfG+7IjxP+GfD3/FCJshrXUEluiMUP8tdwNnBYSinLkJLqfs3a072dBLyWH1Yzcs26lNLf83XfSO578U9yNweuubHyMHK9vvPJhfJh+ePdQu69fZLcWO0PyH1w2RiXkPurwTxyYfh31da1Bf6H3Bj2f5O7OfFHkPsgQO6m1OrbS9pERW64niSpIeSHHCwChqWUntzQ9i1RRGxB7kPE/i3lYSr5qfc6pZR+WOxaJBWfAVqSPqWIGExuyMHHwIXAd4EvpJTW1wstSWqmHMIhSZ/efuSGErxF7s/8RxmeJanlsgdakiRJysAeaEmSJCkDA7QkSZKUQetiF5DVdtttl7p3717sMiRJktTCzZgx462U0jpz+ze7AN29e3fKy8uLXYYkSZJauIh4rbZ2h3BIkiRJGRigJUmSpAwM0JIkSVIGBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZIkKQMDtCRJkpSBAVqSJEnKwAAtSZIkZWCAliRJkjIwQEuSpAY3f/58DjzwQEpKSujZsyfXXnstAKNHj6ZPnz6UlpZy8MEHs2jRolr332yzzSgtLaW0tJQjjjiiMUuXNqhRAnREdIuIv0bEnIh4PiLOzLf/OCIWRkRF/mtIY9RTDJ/2FwnA+++/zw477MBpp53WWGVLzZ4/e1JxtG7dmiuvvJI5c+YwdepUbrjhBl544QXOO+88Zs2aRUVFBYcddhhjxoypdf8tt9ySiooKKioqeOihhxq5emn9GqsHeiVwbkqpBOgP/HdE7JFfd3VKqTT/NbGR6ml0n/YXCeT+h7///vs3YtVS8+fPnlQcXbp0Yc899wSgffv2lJSUsHDhQjp06FC1zUcffUREFKtEqd4aJUCnlBanlJ7Nv/4AmAPs0Bjnbio+7S+SGTNm8MYbb3DwwQc3Sr1SS+HPnlR88+bNY+bMmeyzzz4AXHTRRXTr1o3bb7+9zg+vH3/8MWVlZfTv358HHnigMcuVNqjRx0BHRHegH/D3fNNpETErIm6NiG0bu55iyPqLZPXq1Zx77rlcccUVjV2q1KL4syc1vg8//JChQ4dyzTXXVH1w/clPfsL8+fM5/vjjuf7662vd7/XXX6e8vJw77riDs846i1deeaUxy5bWq1EDdERsBdwHnJVSeh+4EdgFKAUWA1fWsd8pEVEeEeWVlZWNVm8h1OcXyf/+7/8yZMgQunXr1tjlSi2GP3tS41uxYgVDhw7l+OOP55hjjlln/XHHHcd9991X677bb789AF/4whc44IADmDlzZkFrlbKIlFLjnCiiDfAnYFJK6apa1ncH/pRS6rW+45SVlaXy8vKC1FhoK1as4LDDDmPQoEGcc84566x/7bXXOPTQQ5k9e/Za7ccffzxPPvkkrVq14sMPP2T58uWceuqp/PznP2+s0qVmzZ89qfGllDjppJPo2LEj11xzTVX7Sy+9RI8ePQC47rrrePzxx/n973+/1r5LliyhXbt2tG3blrfeeosBAwbw4IMPssceeyA1poiYkVIqq9neupFOHsBvgDnVw3NEdEkpLc4vHg3Mrm3/liClxMiRIykpKVnrf+DVf5E89NBD7L777uvse/vtt1e9HjduHOXl5f4PXNpI/uxJxfH0008zYcIEevfuTWlpKQA//elP+c1vfsPcuXNp1aoVO+20EzfddBMA5eXl3HTTTYwdO5Y5c+bwve99j1atWrF69WouuOACw7OalEYJ0MC+wAnAPyKiIt/2Q2BERJQCCZgHfK+R6ml0n+YXiaT682dPKo799tuP2v7KPWRI7TPWlpWVVf3cffnLX+Yf//hHQeuTPo1GG8LRUJrzEA5JkiQ1H3UN4fBJhJIkSVIGBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRl0FjT2EmSpGag+wV/LnYJBTXv54cWuwS1AAboDfAXiVQc/uxJkpoqh3BIkiRJGRigJUmSpAwM0JIkSVIGBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZIkKQMDtCRJkpSBAVqSJEnKwAAtSZIkZWCAliRJkjIwQEuSJEkZGKAlSZKkDAzQkiRJUgaNEqAjoltE/DUi5kTE8xFxZr69Y0T8X0S8lP9328aoR5IkSaqvxuqBXgmcm1IqAfoD/x0RewAXAJNTSj2AyfllSZIkqclqlACdUlqcUno2//oDYA6wA3AkMD6/2XjgqMaoR5IkSaqvRh8DHRHdgX7A34HPpZQWQy5kA50bux5JkiQpi0YN0BGxFXAfcFZK6f0M+50SEeURUV5ZWVm4AiVJkqQNaLQAHRFtyIXn21NKf8g3vxERXfLruwBv1rZvSunmlFJZSqmsU6dOjVOwJEmSVIvGmoUjgN8Ac1JKV1Vb9RBwUv71ScCDjVGPJEmSVF+tG+k8+wInAP+IiIp82w+BnwP3RMRI4HXg2EaqR5IkSaqXRgnQKaWngKhj9dcaowZJkiSpIfgkQkmSJCkDA7QkSZKUgQFakiRJazn55JPp3LkzvXr1Wqv9uuuu44tf/CI9e/bk/PPPr3Xfa6+9ll69etGzZ0+uueaaxii30RmgJUmStJZvfetbPPLII2u1/fWvf+XBBx9k1qxZPP/884waNWqd/WbPns0tt9zCtGnTeO655/jTn/7ESy+91FhlNxoDtCRJktby1a9+lY4dO67VduONN3LBBRfQtm1bADp3XvcB0nPmzKF///60a9eO1q1bs//++3P//fc3Ss2NyQAtSZKkDfrnP//Jk08+yT777MP+++/P9OnT19mmV69ePPHEE7z99tssXbqUiRMnMn/+/CJUW1gGaGkD6jsO7OOPP2bvvfemb9++9OzZk0suuaSxSpYkqcGtXLmSJUuWMHXqVK644gqGDx9OSmmtbUpKSvjBD37AwIEDGTx4MH379qV168Z67EjjMUBLG1DfcWBt27ZlypQpPPfcc1RUVPDII48wderUxipbahHq+wF2/vz5HHjggZSUlNCzZ0+uvfbaxipZarG6du3KMcccQ0Sw995706pVK9566611ths5ciTPPvssTzzxBB07dqRHjx5FqLawDNDSBtR3HFhEsNVWWwGwYsUKVqxYQe6p9pI2Vn0/wLZu3Zorr7ySOXPmMHXqVG644QZeeOGFxipbapGOOuoopkyZAuSGcyxfvpzttttune3efPNNAF5//XX+8Ic/MGLEiEatszEYoKV62JhxYACrVq2itLSUzp07M3DgQPbZZ59GrlRq3ur7AbZLly7sueeeALRv356SkhIWLlxY+IKlFmLEiBEMGDCAuXPn0rVrV37zm99w8skn8+qrr9KrVy+++c1vMn78eCKCRYsWMWTIkKp9hw4dyh577MHhhx/ODTfcwLbbblvEKymMljcoRWoE1ceBTZ8+neHDh/Pqq6+u08O82WabUVFRwbvvvsvRRx/N7Nmz1/lTtKRs1nyAveiii9hiiy345S9/yV577VXn9vPmzWPmzJl+gJUyuPPOO2ttv+2229Zp23777Zk4cWLV8pNPPlmwupoKe6CletjYcWBrbLPNNhxwwAHr/ClaUnYbcyPTGh9++CFDhw7lmmuuoUOHDo1cqaSWygAt1cPGjAOrrKzk3XffBWDZsmU8+uij7L777o1eq9TSbOwH2BUrVjB06FCOP/54jjnmmCJUKqmlMkBLG1DfcWCLFy/mwAMPpE+fPuy1114MHDiQww47rMhXIzV/G/MBNqXEyJEjKSkp4ZxzzilGmZJaMMdASxtQ33Fgffr0YebMmQWtTWrpRowYwWOPPcZbb71F165dufTSSzn55JM5+eST6dWrF5tvvvlaH2C/853vMHHiRJ5++mkmTJhA7969KS0tBeCnP/3pWjc6SVJ9GaAlSU1WfT/A7rfffnWOi5akT8sALUmS1EJ0v+DPxS6hoOb9/NBilwDUYwx0RHwmIlrlX+8WEUdERJuGL02SJElqeupzE+ETwBYRsQMwGfg2MK4hi5IkSZKaqvoE6EgpLQWOAa5LKR0N7NGwZUmSJElNU70CdEQMAI4H1gy0cSy1JEmSNgn1Cb5nARcC96eUno+ILwB/bdiypIbRkm+maCo3Uki1ack/e+DPn7SpyxygU0qPA49XW34VOKMhi5IkSZKaqo0O0BHxR6DOSTVTSkc0SEWSJElSE5alB/qX+X+PAT4PrJnFfgQwrwFrkiRJkpqsjQ7Q+aEbRMRlKaWvVlv1x4h4osErkyRJkpqg+szC0Sl/4yAAEbEz0Gl9O0TErRHxZkTMrtb244hYGBEV+a8h9ahFkiRJalT1mYXjbOCxiHg1v9wd+N4G9hkHXA/8rkb71SmlX667uSRJktQ01WcWjkciogewe77pxZTSJxvY54mI6J69PEmSJKlpqc8QDoAvAT2BvsA3IuLEeh7ntIiYlR/isW09jyFJkiQ1mswBOiImkJuRYz9gr/xXWT3OfSOwC1AKLAauXM85T4mI8ogor6ysrMepJEmSpIZRnzHQZcAeKaU654TeGCmlN9a8johbgD+tZ9ubgZsBysrKPtV5JUmSpE+jPkM4ZpObB/pTiYgu1RaPzh9XkiRJatLq0wO9HfBCREwDqm4eXN+TCCPiTuAAYLuIWABcAhwQEaXknm44jw3P5CFJkiQVXX0C9I+z7pBSGlFL82/qcW5JkiSpqOozjd3jEfE5cjcPAkxLKb3ZsGVJkiRJTVN9ZuEYDkwDjgWGA3+PiGENXZgkSZLUFNVnCMdFwF5rep0johPwKPD7hixMkiRJaorqMwtHqxpDNt6u53EkSZKkZqc+PdCPRMQk4M788jeAhxuuJEmSJKnpqs9NhOdFxDHknkQYwM0ppfsbvDJJkiSpCcocoCNiZ2BiSukP+eUtI6J7SmleQxcnSZIkNTX1Gbt8L7C62vKqfJskSZLU4tUnQLdOKS1fs5B/vXnDlSRJkiQ1XfUJ0JURUfXY7og4Enir4UqSJEmSmq76zMLxfeD2iLgBSMAC4MQGrUqSJElqouozC8crQP+I2AqIlNIHDV+WJEmS1DTV51Hen4uI3wD3ppQ+iIg9ImJkAWqTJEmSmpz6jIEeB0wCts8v/xM4q6EKkiRJkpqy+gTo7VJK95Cfyi6ltJLcVHaSJElSi1efAP1RRHyW3A2ERER/4L0GrUqSJElqouozC8c5wEPALhHxNNAJGNagVUmSJElNVH16oHcBDgG+TG4s9EvUL4hLkiRJzU59AvTolNL7wLbA14GbgRsbtCpJkiSpiapPgF5zw+ChwE0ppQfxUd6SJEnaRNQnQC+MiF8Dw4GJEdG2nseRJEmSmp36BN/h5MY+D04pvQt0BM5r0KokSZKkJqo+j/JeCvyh2vJiYHFDFiVJkiQ1VQ69kCRJkjIwQEuSJEkZGKAlSZKkDBotQEfErRHxZkTMrtbWMSL+LyJeyv+7bWPVI0mSJNVHY/ZAjwMG12i7AJicUuoBTM4vS5IkSU1WowXolNITwDs1mo8ExudfjweOaqx6JEmSpPoo9hjoz+WnwVszHV7n2jaKiFMiojwiyisrKxu1QEmSJKm6YgfojZJSujmlVJZSKuvUqVOxy5EkSdImrNgB+o2I6AKQ//fNItcjSZIkrVexA/RDwEn51ycBDxaxFkmSJGmDGnMauzuBvwFfjIgFETES+DkwMCJeAgbmlyVJkqQmq3VjnSilNKKOVV9rrBokSZKkT6vYQzgkSZKkZsUALUmSJGVggJYkSZIyMEBLkiRJGRigJUmSpAwM0JIkSVIGBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZIkKQMDtCRJkpSBAVqSJEnKwAAtSZIkZWCAliRJkjIwQEuSJEkZGKAlSZKkDAzQkiRJUgYGaEmSJCkDA7QkSZKUgQFakiRJysAALUmSJGXQutgFAETEPOADYBWwMqVUVtyKJEmSpNo1iQCdd2BK6a1iFyFJkiStj0M4JEmSpAyaSoBOwF8iYkZEnFJzZUScEhHlEVFeWVlZhPIkSZKknKYSoPdNKe0JHAL8d0R8tfrKlNLNKaWylFJZp06dilOhJEmSRBMJ0CmlRfl/3wTuB/YubkWSJElS7YoeoCPiMxHRfs1r4GBgdnGrkiRJkmrXFGbh+Bxwf0RArp47UkqPFLckSZIkqXZFD9AppVeBvsWuQ5IkSdoYRR/CIUmSJDUnBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZIkKQMDtCRJkpSBAVqSJEnKwAAtSZIkZWCAliRJkjIwQEuSJEkZGKAlSZKkDAzQkiRJUgYGaEmSJCkDA7QkSZKUgQFakiRJysAALUmSJGVggJYkSZIyMEBLkiRJGRigJUmSpAwM0JIkSVIGBmhJkiQpg6IH6IgYHBFzI+LliLig2PVIkiRJ61PUAB0RmwE3AIcAewAjImKPYtYkSZIkrU+xe6D3Bl5OKb2aUloO3AUcWeSaJEmSpDpFSql4J48YBgxOKX0nv3wCsE9K6bQa250CnJJf/CIwt1ELbVzbAW8VuwjVi+9d8+b717z5/jVfvnfNW0t//3ZKKXWq2di6GJVUE7W0rZPoU0o3AzcXvpzii4jylFJZsetQdr53zZvvX/Pm+9d8+d41b5vq+1fsIRwLgG7VlrsCi4pUiyRJkrRBxQ7Q04EeEbFzRGwOfBN4qMg1SZIkSXUq6hCOlNLKiDgNmARsBtyaUnq+mDU1AZvEUJUWyveuefP9a958/5ov37vmbZN8/4p6E6EkSZLU3BR7CIckSZLUrBigJUmSpAwM0JIkSVIGBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZIkKQMDtCRJkpSBAVqSJEnKwAAtSZIkZWCAliRJkjIwQEuSJEkZGKAlSZKkDAzQkqR6iYg7I+KoeuzXNiJejIjOhahLkgrNAC1JDSwiHouIJRHRtti1FEpE9AH6Ag/ml/tGxPMR8VZEnF1tuzYR8feI6LamLaX0CXAr8IPGrluSGoIBWpIaUER0B74CJOCIRj5360Y83feA21NKKb/8M2AUuVD9o4j4fL79HOC+lNL8GvvfAZzUkj9kSGq5DNCS1LBOBKYC44CTqq+IiC0j4sqIeC0i3ouIpyJiy/y6/SLimYh4NyLmR8S38u2PRcR3qh3jWxHxVLXlFBH/HREvAS/l267NH+P9iJgREV+ptv1mEfHDiHglIj7Ir+8WETdExJU16v1jRJxVx3UeAjxebXlnYEpKaWG+jh0jYkdgKHB1zZ1TSguAJUD/9X0zJakpMkBLUsM6Ebg9/zUoIj5Xbd0vgS8BXwY6AucDq/NB82HgOqATUApUZDjnUcA+wB755en5Y3Qk19N7b0RskV93DjACGAJ0AE4GlgLjgRER0QogIrYDvgbcWfNkEfEZcoF5brXm2cDBEdEV6A68AvwKOD+ltKKOuueQ67GWpGbFAC1JDSQi9gN2Au5JKc0gFyKPy69rRS6snplSWphSWpVSeiY/Hvh44NGU0p0ppRUppbdTSlkC9M9SSu+klJYBpJRuyx9jZUrpSqAt8MX8tt8BfpRSmptynstvOw14j1xoBvgm8FhK6Y1azrdN/t8PqrWNAv4f8BBwNrBvfv2rEfFgRDweEcfWOM4H1Y4lSc2GAVqSGs5JwF9SSm/ll+/gP8M4tgO2IBeqa+pWR/vGWmt8cUScGxFz8sNE3gW2zp9/Q+caD/xX/vV/ARPq2O7d/L/t1zSklF5LKQ1JKe1J7sbCMeRC9S+Bu8mNB78qIjpWO077aseSpGbDAC1JDSA/lnk4sH9E/Dsi/k2uJ7ZvRPQF3gI+BnapZff5dbQDfAS0q7b8+Vq2WXMjH/nxzj/I17JtSmkbcj3LsRHnug04Ml9vCfBAbRullD4iF8J3q+M4FwNj873XvYHylNJ7wAJg12rblQDP1XEMSWqyDNCS1DCOAlaRG4dcmv8qAZ4ETkwprSY3ddtVEbF9/ma+AflZKG4Hvh4RwyOidUR8NiJK88etAI6JiHYRsSswcgN1tAdWApVA64i4mNxY5zXGApdFRI/I6RMRn4WqG/umk+t5vm/NkJA6TAT2r9kYEXsABwA35pv+BRyUHwveA3g9v90O5MZoT93A9UhSk2OAlqSGcRLw25TS6ymlf6/5Aq4Hjs9PMTcK+Ae5kPoO8AugVUrpdXI39Z2bb6/gPzfXXQ0sB94gN8Ti9g3UMYncDYn/BF4j1+tdfYjHVcA9wF+A94HfAFtWWz+eXK9xXcM31rg5f11Ro/0GcuO8V+WXLwTOAJ4Hfpr/nkBubPj4/BhwSWpW4j9TeEqSNnUR8VVyQzm653vN17ftHeRumKx1qMd69mtLbujGV1NKb9a7WEkqEgO0JAnIPTUQuAt4LqU0ptj1SFJT5RAOSRIRUUJuRowuwDVFLkeSmjR7oCVJkqQM7IGWJEmSMjBAS5IkSRm0LnYBWW233Xape5vQEhoAACAASURBVPfuxS5DkiRJLdyMGTPeSil1qtne7AJ09+7dKS8vL3YZkiRJauEi4rXa2h3CIUmSJGVggJYkSZIyMEBLkiRJGRigJUmSpAwM0JIkSVIGBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRlULAAHRG3RsSbETG7jvUREb+KiJcjYlZE7FmoWiRJkqSGUsge6HHA4PWsPwTokf86BbixgLVIkiRJDaJgATql9ATwzno2ORL4XcqZCmwTEV0KVY8kSZLUEIo5BnoHYH615QX5tnVExCkRUR4R5ZWVlY1SnCRJklSbYgboqKUt1bZhSunmlFJZSqmsU6dOBS5LkiRJqlsxA/QCoFu15a7AoiLVIkmSJG2UYgboh4AT87Nx9AfeSyktLmI9kiRJ0gYVchq7O4G/AV+MiAURMTIivh8R389vMhF4FXgZuAU4tVC1SJIkaW1XX301PXv2pFevXowYMYKPP/6YKVOmsOeee9KrVy9OOukkVq5cuc5+f/3rXyktLa362mKLLXjggQcA+MpXvlLVvv3223PUUUcBcN9999GzZ0++8pWv8PbbbwPwyiuv8M1vfrPxLrgBRUq1DjtussrKylJ5eXmxy5AkSWq2Fi5cyH777ccLL7zAlltuyfDhwxk8eDCXXHIJkydPZrfdduPiiy9mp512YuTIkXUe55133mHXXXdlwYIFtGvXbq11Q4cO5cgjj+TEE0/ky1/+MpMmTeKuu+7i448/5vTTT2fEiBGMGTOGHj16FPpy6y0iZqSUymq2+yRCSZKkTdDKlStZtmwZK1euZOnSpXzmM5+hbdu27LbbbgAMHDiQ++67b73H+P3vf88hhxyyTnj+4IMPmDJlSlUPdKtWrfjkk09YunQpbdq04cknn6RLly5NOjyvjwFakiRpE7PDDjswatQodtxxR7p06cLWW2/N8OHDWbFiBWv+0v/73/+e+fPnr/c4d911FyNGjFin/f777+drX/saHTp0AOCSSy5h0KBBPProo4wYMYLLL7+c0aNHN/yFNRIDtCRJ0iZmyZIlPPjgg/zrX/9i0aJFfPTRR9x+++3cddddnH322ey99960b9+e1q1b13mMxYsX849//INBgwats+7OO+9cK1gPHDiQGTNm8Mc//pEHHniAIUOGMHfuXIYNG8Z3v/tdli5dWpDrLBQDtCRJ0ibm0UcfZeedd6ZTp060adOGY445hmeeeYYBAwbw5JNPMm3aNL761a+ud4jFPffcw9FHH02bNm3Wan/77beZNm0ahx566Dr7LF26lPHjx3Pqqady4YUXcuutt/KlL32J22+/vcGvsZAM0JIkqV5qm8UhpcRFF13EbrvtRklJCb/61a/W2W99sziscfrpp7PVVltVLV933XX06tWLIUOGsHz5cgCeeuopzjnnnMJeZAu14447MnXqVJYuXUpKicmTJ1NSUsKbb74JwCeffMIvfvELvv/979d5jJq9zGvce++9HHbYYWyxxRbrrPuf//kfzjzzTNq0acOyZcuICFq1amUPtCRJavkWLlzIr371K8rLy5k9ezarVq3irrvuYty4ccyfP58XX3yROXPm1DpN2YEHHkhFRQUVFRVMmTKFdu3acfDBB1etLy8v5913311rn7FjxzJr1iz69evHpEmTSClx2WWXNetxtMW0zz77MGzYMPbcc0969+7N6tWrOeWUU7jiiisoKSmhT58+HH744Rx00EFA7j35zne+U7X/vHnzmD9/Pvvvv/86x65rXPSiRYsoLy/nyCOPBODcc8+lf//+jB8/nuOOO65AV1oYTmMnSZIyW7hwIf379+e5556jQ4cOHHXUUZxxxhn86Ec/4o477mDXXXfdqOPcfPPNPP7441V/wl+1ahVf//rXueOOO+jRowcffvghAH379mXatGlccMEFDBo0iDfffJMlS5Zw5plnFuwaJaexkyRJDaa2WRwOPvhgXnnlFe6++27Kyso45JBDeOmll9Z7nJq9lddffz1HHHEEXbp0WWu7UaNG0b9/fyorK9l3332rxtFKxWCAliRJmdU2i8Ntt93GJ598whZbbEF5eTnf/e53Ofnkk+s8Rs1ZHBYtWsS9997L6aefvs62J5xwAjNnzuS2227jqquu4owzzuDhhx9m2LBhnH322axevbpg1yrVZICWJEmZ1TWLQ9euXRk6dCgARx99NLNmzarzGDVncZg5cyYvv/wyu+66K927d2fp0qXrDAVZtGgR06dP58gjj+Tyyy/n7rvvpm3btkyePLlwFyvVYICWJEmZ1TWLw1FHHcWUKVMAePzxx6uealebmrM4HHroofz73/9m3rx5zJs3j3bt2vHyyy+vtc/o0aO57LLLAJr1LA5q3gzQkiQps7pmcbjgggu477776N27NxdeeCFjx44Fss3iUJeZM2cC0K9fPwBGjhxJ7969efbZZxk8eHADXp20fs7CIUmSJNWirlk46n4+oyRJkpqXH29d7AoK68fvFbsCwCEckiRJUiYGaEmSJCkDA7QkSZKUgQFakiRJysAALUmSJGXgLBySJOk/nMVB2iB7oCVJRTN37lxKS0urvjp06MA111xDRUUF/fv3p7S0lLKyMqZNm1br/uPHj6dHjx706NGD8ePHV7UPHjyYvn370rNnT77//e+zatUqAH7wgx/Qp08fTjzxxKptJ0yYwLXXXlvYC5XUohigJUlF88UvfpGKigoqKiqYMWMG7dq14+ijj+b888/nkksuoaKigjFjxnD++eevs+8777zDpZdeyt///nemTZvGpZdeypIlSwC45557eO6555g9ezaVlZXce++9vPfeezzzzDPMmjWLVatW8Y9//INly5Yxbtw4Tj311Ma+dEnNmAFaktQkTJ48mV122YWddtqJiOD9998H4L333mP77bdfZ/tJkyYxcOBAOnbsyLbbbsvAgQN55JFHAOjQoQMAK1euZPny5UQErVq1Yvny5aSUWLZsGW3atOGKK67gjDPOoE2bNo13oZKaPQO0JKlJuOuuuxgxYgQA11xzDeeddx7dunVj1KhR/OxnP1tn+4ULF9KtW7eq5a5du7Jw4cKq5UGDBtG5c2fat2/PsGHDaN++PUOHDqVfv37svPPObL311kyfPp0jjzyy8BcnqUUxQEuSim758uU89NBDHHvssQDceOONXH311cyfP5+rr76akSNHrrNPSmmdtoioej1p0iQWL17MJ598wpQpUwA4//zzqaio4Morr2T06NGMGTOGsWPHMnz4cC6//PICXZ2klsYALUkquocffpg999yTz33uc0Du5sBjjjkGgGOPPbbWmwi7du3K/Pnzq5YXLFiwzlCPLbbYgiOOOIIHH3xwrfaZM2cCsNtuu/G73/2Oe+65h9mzZ/PSSy816HVJapkM0JKkorvzzjurhm8AbL/99jz++OMATJkyhR49eqyzz6BBg/jLX/7CkiVLWLJkCX/5y18YNGgQH374IYsXLwZyY6AnTpzI7rvvvta+a3qfV6xYUTVDR6tWrVi6dGmhLlFSC1LQeaAjYjBwLbAZMDal9PMa63cExgPb5Le5IKU0sZA1SZKalqVLl/J///d//PrXv65qu+WWWzjzzDNZuXIlW2yxBTfffDMA5eXl3HTTTYwdO5aOHTsyevRo9tprLwAuvvhiOnbsyBtvvMERRxzBJ598wqpVqzjooIP4/ve/X3XsBx54gL322quqt3rAgAH07t2bPn360Ldv30a8cknNVdQ2hqxBDhyxGfBPYCCwAJgOjEgpvVBtm5uBmSmlGyNiD2BiSqn7+o5bVlaWysvLC1KzpOZn7ty5fOMb36hafvXVVxkzZgxvv/02Dz74IK1ataJz586MGzeu1pkczj//fP785z+zevVqBg4cyLXXXktEsHz5ck477TQee+wxWrVqxU9+8hOGDh3Kddddx69//Wt23HFHHnjgATbffHOeeuop/vCHP3DVVVc15qVLheGDVJo3378GFREzUkplNdsLOYRjb+DllNKrKaXlwF1AzVudE9Ah/3prYFEB65HUAtU1j/B5553HrFmzqKio4LDDDmPMmDHr7PvMM8/w9NNPM2vWLGbPns306dOrhg385Cc/oXPnzvzzn//khRdeYP/99wdg7NixzJo1i379+jFp0iRSSlx22WWMHj26Ua9bklQ8hRzCsQMwv9ryAmCfGtv8GPhLRJwOfAb4egHrkdTCVZ9HuLqPPvpordkZ1ogIPv7446q5gVesWFF1E9utt97Kiy++COTGxm633XZV+61YsYKlS5fSpk0bJkyYwJAhQ9h2220LeGWSpKakkD3Q6/7fKtfjXN0IYFxKqSswBJgQEevUFBGnRER5RJRXVlYWoFRJLUH1eYQBLrroIrp168btt99eaw/0gAEDOPDAA+nSpQtdunRh0KBBlJSU8O677wK5G8323HNPjj32WN544w0ARo0aRf/+/amsrGTfffdl/PjxPsVOkjYxhQzQC4Bu1Za7su4QjZHAPQAppb8BWwDb1diGlNLNKaWylFJZp06dClSupOas5jzCkBuGMX/+fI4//niuv/76dfZ5+eWXmTNnDgsWLGDhwoVMmTKFJ554gpUrV7JgwQL23Xdfnn32WQYMGMCoUaMAOOGEE5g5cya33XYbV111FWeccQYPP/www4YN4+yzz2b16tWNds2SpOIoZICeDvSIiJ0jYnPgm8BDNbZ5HfgaQESUkAvQdjFLyqzmPMLVHXfccdx3333rtN9///3079+frbbaiq222opDDjmEqVOn8tnPfrZqLDXk5iF+9tln19p30aJFVU+xu/zyy7n77rtp27YtkydPLswFSpKajIKNgU4prYyI04BJ5KaouzWl9HxEjAHKU0oPAecCt0TE2eSGd3wrFWpaEEktWs15hF966aWquYMfeuihdeYBBthxxx255ZZbuPDCC0kp8fjjj3PWWWcRERx++OE89thjHHTQQUyePJk99thjrX1Hjx7NZZddBsCyZcuICOcRrs6ZACS1YAWdBzo/p/PEGm0XV3v9ArBvIWuQ1PLVNo/wBRdcwNy5c2nVqhU77bQTN910E7D2PMLDhg1jypQp9O7dm4hg8ODBHH744QD84he/4IQTTuCss86iU6dO/Pa3v6069pqn2PXr1w+AkSNH0rt3b7p168Yll1zSWJctSSqSgs0DXSjOAy1JzYA90M2X713z5vvXoIoxD7QkSZLU4higJUmSpAwM0JIkSVIGBmhJkiQpAwO0JEmSlEFBp7GTpHrzTnJJUhNlD7QkSZKUgQFakiRJysAALUmSJGVggJYkSZIyMEBLkiRJGRigJUmSpAwM0JIkSVIGBmhJkiQpAwO0JEmSlIEBWpIkScrAAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZIkKQMDtCRJkpSBAVqSJEnKwAAtSZIkZWCAliRJkjIwQEuSJEkZFDRAR8TgiJgbES9HxAV1bDM8Il6IiOcj4o5C1iNJkiR9Wq0LdeCI2Ay4ARgILACmR8RDKaUXqm3TA7gQ2DeltCQiOheqHkmSJKkhFLIHem/g5ZTSqyml5cBdwJE1tvkucENKaQlASunNAtYjSZIkfWqFDNA7APOrLS/It1W3G7BbRDwdEVMjYnAB65EkSZI+tYIN4QCilrZUy/l7AAcAXYEnI6JXSundtQ4UcQpwCsCOO+7Y8JVKkiRJG6mQPdALgG7VlrsCi2rZ5sGU0oqU0r+AueQC9VpSSjenlMpSSmWdOnUqWMGSJEnShhQyQE8HekTEzhGxOfBN4KEa2zwAHAgQEduRG9LxagFrkiRJkj6VggXolNJK4DRgEjAHuCel9HxEjImII/KbTQLejogXgL8C56WU3i5UTZIkSdKnVcgx0KSUJgITa7RdXO11As7Jf0mSJElNnk8ilCRJkjIwQEuSJEkZGKAlSZKkDDIF6IjoHxFT8g8+OapQRUmSJElN1XpvIoyIz6eU/l2t6RzgCHIPSXmG3DR0kiRJ0iZjQ7Nw3BQRM4ArUkofA+8CxwGrgfcLXZwkSZLU1Kx3CEdK6SigAvhTRJwAnEUuPLcDHMIhSZKkTc4Gx0CnlP4IDAK2Af4AzE0p/SqlVFno4iRJkqSmZr0BOiKOiIingCnAbHKP4z46Iu6MiF0ao0BJkiSpKdnQGOjLgQHAlsDElNLewDkR0QP4CblALUmSJG0yNhSg3yMXkrcE3lzTmFJ6CcOzJEmSNkEbGgN9NLkbBleSm31DkiRJ2qSttwc6pfQWcF0j1SJJkiQ1eT7KW5IkScrAAC1JkiRlYICWJEmSMjBAS5IkSRkYoCVJkqQMDNCSJElSBgZoSZIkKQMDtCRJkpSBAVqSJEnKwAAtSZIkZWCAliRJkjIwQEuSJEkZGKC1yfv444/Ze++96du3Lz179uSSSy4B4Fvf+hY777wzpaWllJaWUlFRsc6+FRUVDBgwgJ49e9KnTx/uvvvuqnUjR46kb9++9OnTh2HDhvHhhx8CcN1119GrVy+GDBnC8uXLAXjqqac455xzGuFqJUnSp2WA1iavbdu2TJkyheeee46KigoeeeQRpk6dCsAVV1xBRUUFFRUVlJaWrrNvu3bt+N3vfsfzzz/PI488wllnncW7774LwNVXX81zzz3HrFmz2HHHHbn++usBGDt2LLNmzaJfv35MmjSJlBKXXXYZo0ePbryLliRJ9da62AVIxRYRbLXVVgCsWLGCFStWEBEbte9uu+1W9Xr77benc+fOVFZWss0229ChQwcAUkosW7ZsrWOuWLGCpUuX0qZNGyZMmMCQIUPYdtttG/CqJElSoRS0BzoiBkfE3Ih4OSIuWM92wyIiRURZIeuR6rJq1SpKS0vp3LkzAwcOZJ999gHgoosuok+fPpx99tl88skn6z3GtGnTWL58ObvssktV27e//W0+//nP8+KLL3L66acDMGrUKPr3709lZSX77rsv48eP59RTTy3cxUmSpAZVsAAdEZsBNwCHAHsAIyJij1q2aw+cAfy9ULVIG7LZZptRUVHBggULmDZtGrNnz+ZnP/sZL774ItOnT+edd97hF7/4RZ37L168mBNOOIHf/va3tGr1nx+r3/72tyxatIiSkpKq8dEnnHACM2fO5LbbbuOqq67ijDPO4OGHH2bYsGGcffbZrF69uuDXK0mS6q+QPdB7Ay+nlF5NKS0H7gKOrGW7y4D/AT4uYC3SRtlmm2044IADeOSRR+jSpQsRQdu2bfn2t7/NtGnTat3n/fff59BDD+Xyyy+nf//+66zfbLPN+MY3vsF99923VvuiRYuYPn06Rx55JJdffjl33303bdu2ZfLkyQW5NkmS1DAKGaB3AOZXW16Qb6sSEf2AbimlP63vQBFxSkSUR0R5ZWVlw1eqTVplZWXVjX/Lli3j0UcfZffdd2fx4sVAbgzzAw88QK9evdbZd/ny5Rx99NGceOKJHHvssVXtKSVefvnlqtd//OMf2X333dfad/To0Vx22WVV540IWrVqxdKlSwtynZIkqWEU8ibC2u7CSlUrI1oBVwPf2tCBUko3AzcDlJWVpQ1sLmWyePFiTjrpJFatWsXq1asZPnw4hx12GAcddBCVlZWklCgtLeWmm24CoLy8nJtuuomxY8dyzz338MQTT/D2228zbtw4AMaNG0efPn046aSTeP/990kp0bdvX2688caqc86cOROAfv36Abkp73r37k23bt2qptGTJElNU6RUmDwaEQOAH6eUBuWXLwRIKf0sv7w18ArwYX6XzwPvAEeklMrrOm5ZWVkqL69ztaSW4sdbF7uCwvrxe8WuoLB8/5ov37vmzfevQUXEjJTSOpNcFHIIx3SgR0TsHBGbA98EHlqzMqX0Xkppu5RS95RSd2AqGwjPkiRJUrEVLECnlFYCpwGTgDnAPSml5yNiTEQcUajzSpIkSYVU0AeppJQmAhNrtF1cx7YHFLIWSZIkqSH4KG9JkiQpAwN0A5k/fz4HHnggJSUl9OzZk2uvvRaAiooK+vfvT2lpKWVlZXXOJQy5+YR32GEHTjvttKq2iy66iG7dulU9anqN6667jl69ejFkyBCWL18OwFNPPcU555xTgKuTJEnSGgboBtK6dWuuvPJK5syZw9SpU7nhhht44YUXOP/887nkkkuoqKhgzJgxnH/++XUeY/To0ey///5rtR1++OG1hu6xY8cya9Ys+vXrx6RJk0gpcdlllzF69OgGvzZJkiT9R0HHQG9KunTpQpcuXQBo3749JSUlLFy4kIjg/fffB+C9995j++23r3X/GTNm8MYbbzB48GCqT9NX25Pt1lixYgVLly6lTZs2TJgwgSFDhrDttts24FW1AC15Op+WPhWTJElNlAG6AObNm8fMmTPZZ599uOaaaxg0aBCjRo1i9erVPPPMM+tsv3r1as4991wmTJiw0Y9xHjVqFP3796dnz57su+++HHXUUf+/vTuPtrQq7zz+/TEpDhhFNEFmQQOGMQTTxiga45QgCYgalWhLNL0SwSHRNqhpgnaaGIMkGo0gCKJxwNiKBsWOBJI4RGaQSZHgENA4oQwi09N/nFPUvWXVveet4t599j3fz1q16p73vFX11PpxN0/ts/d++dSnPnVP/1UkSZK0Bpdw3MNuuukmDj74YI477ji22GIL3vGOd/CWt7yFb3zjG7zlLW/hsMMO+6lf8/a3v52nP/3pbLvtthP/OYceeigXXngh733vezn22GM54ogj+OQnP8kzn/lMXvGKV3DXXXfdk38tSZIkjdlA34Nuv/12Dj74YJ73vOdx0EEHAXDKKafc/fUhhxyy1vXMn//853nb297GDjvswB//8R/znve8h9e85jUT/ZnXXXcd5557LgceeCBvfOMb+eAHP8i97nWviWeyJUmSNIxLOO4hVcVhhx3GrrvuOu8kjK233ppzzjmH/fffn7POOotddtnlp37t+973vru/PvnkkznvvPM45phjJvpzX//61/OGN7wBgB//+MckYaONNuKWW27ZwL+RJEmS1sYZ6HvIZz/7WU499VTOOuss9tprL/baay/OOOMMTjjhBP7oj/6IPffckyOPPJLjjz8egPPOO4/f+73fW/T3ffWrX80222zDLbfcwjbbbMNRRx1193sXXnghAHvvvTcAhx12GLvvvjsXXHABT33qU+/5v6QkSZJIVbWuYZB999235p5SIS3IUzj6tZKzA/Pr3UrOz+z6Zn73qCTnV9W+a153BlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrAB6ksxuNgJEmSNIcz0JIkSdIANtCSJEnSADbQkiRJ0gA20JIkSdIANtCSJEnSADbQkiRJ0gA20JIkSdIANtCSJEnSADbQkiRJ0gA20JIkSdIAS9pAJ3lqkquSXJ3kNWt5/5VJLk9ySZLPJNl+KeuRJEmSNtSSNdBJNgb+FngasBvwO0l2W+O2C4F9q2oP4MPAm5aqHkmSJOmesJQz0PsBV1fVNVV1G/AB4MC5N1TVP1fVLeOXXwC2WcJ6JEmSpA22lA30w4BvzHn9zfG1dTkM+OQS1iNJkiRtsE2W8PfOWq7VWm9Mng/sCzx+He+/BHgJwHbbbXdP1SdJkiQNtpQz0N8Etp3zehvgujVvSvIk4LXAM6rqJ2v7jarq+Krat6r23WqrrZakWEmSJGkSS9lAnwvskmTHJJsBzwFOn3tDkr2BdzJqnv9rCWuRJEmS7hFL1kBX1R3AS4EzgSuAD1XVZUmOTvKM8W1/CdwPOC3JRUlOX8dvJ0mSJE2FpVwDTVWdAZyxxrU/nfP1k5byz5ckSZLuaT6JUJIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRrABlqSJEkawAZakiRJGsAGWpIkSRpgSRvoJE9NclWSq5O8Zi3v3yvJB8fv/3uSHZayHkmSJGlDLVkDnWRj4G+BpwG7Ab+TZLc1bjsM+EFV7Qy8BfiLpapHkiRJuics5Qz0fsDVVXVNVd0GfAA4cI17DgROGX/9YeDXkmQJa5IkSZI2yFI20A8DvjHn9TfH19Z6T1XdAfwQ2HIJa5IkSZI2yCZL+HuvbSa51uMekrwEeMn45U1JrtrA2qbZg4HvLtuf9mdO+N+DzK5v5tc38+uX2fVtpee3/douLmUD/U1g2zmvtwGuW8c930yyCfAA4Ptr/kZVdTxw/BLVOVWSnFdV+7auQ8OZXd/Mr2/m1y+z69us5reUSzjOBXZJsmOSzYDnAKevcc/pwAvGXz8TOKuqfmoGWpIkSZoWSzYDXVV3JHkpcCawMXBSVV2W5GjgvKo6HTgRODXJ1Yxmnp+zVPVIkiRJ94SlXMJBVZ0BnLHGtT+d8/WtwCFLWUOHZmKpygpldn0zv76ZX7/Mrm8zmV9cMSFJkiRNzkd5S5IkSQPYQEuSJEkDLOkaaK1bkoMmuO3W8TpyTZkkr5zgtpur6p1LXowGS7LPBLfdXlWXLnkxGsSxU2ojyYMmuO2uqrphyYuZAq6BbiTJ94CPsfaHyazyuKp6+DKVpAGSXA+8g4Xze15VPWKZStIASW5kdNTmQvntWFU7LE9FmpRjZ9+S/GixW4DrHTunT5JbGT3PY6HvvY2rartlKqkpZ6Db+WRVvWihG5K8d7mK0WCnVtXRC92Q5L7LVYwGO7eqnrjQDUnOWq5iNIhjZ9++WlV7L3RDkguXqxgNcoXZreYMtCRJWhZJdqqqazb0Hi2/VQcBjgAAIABJREFUJPceHz+8QfesFDbQDSW5H/BURo8zvwP4CvDpqrqraWGaSJKHA7/N/PzeX1U/bFqYJpZkX+bkV1VXNi5JE3DsXBmSPBC4o6pubF2LJpdkK2AbRt97/1FVNzUuqQlP4WgkybOAf2b0P4GXAvsBhwIXJdm9ZW1aXJKXAX8H3Bv4JWBzRv8z/3yS/RuWpgkkeXyS84BjgJOA3wdOTHJ2km3bVqeFOHb2LcnWSd6T5IfAd4HLknw9yVFJNm1dn9YtyW5J/gn4PPDvwLuAS5OcnOQBbatbfs5AN5LkEuCXq+qWJA8G3ldVT0myB/B3VfWYxiVqAUkuBfaqqjuT3Ac4o6r2T7Id8LHF1omprfE6vSdX1XeS7AgcW1W/neTXgVdV1ZMbl6h1cOzs23hvwdFVdfb4RJVfBV4H/AnwkKp6SdMCtU5JvgC8oKquSrIf8IdV9YIkLwaeUlXPbFzisnIGup0APx5/fTPwEICqugTYolVRGmTVJtx7AfcHqKqvA86iTL+Nq+o746+/DmwPUFX/D3hYs6o0CcfOvm1ZVWcDVNVHGJ2YcnNVvQ54XNPKtJjNq+oqgKr6IrD7+OsTgN1aFtaCp3C0cwbwqSTnAE8DToO7z1lc6IgYTYd3AeeO/0X+OOAv4O61Yd9vWZgmcl6SE4HPAAcCZwOMP03YuGFdWpxjZ9++k+T5wFnAwcC1AEmCk3rT7qtJXs9o3DwIuAhgvPRm5vpJl3A0lOTpjP7VdvF45oskGwGbVtVPmhanRSV5FLAr8CU3n/VlPOC/mPH3H3DSeDnO5ow+Rv5a0wK1IMfOfo2Xub2ZUX4XMVoydX2SLYH9q+ofmhaodUryM8CRrB43j6mqG8frn3etqi80LXCZ2UBPAXci98vdyFI7jp2SWvHjkkbcidw3dyP3Lcn9khyd5EtJfpjkO0m+kOSFrWvTwhw7+5fkCUneluRjSf4hyTFJdm5dlxaWZKMk/z3JJ5JcnOT8JB+Y1ZOnbKDbeS+jj40fABwC/AOj5QCbAH/bsjBN5CRGO5B3Bh4LXFlVOwKfBU5sWpkm8T7gGkZHof0Z8DeMjkJ7QpI/b1mYFuXY2bEkxwC/C3wBuJ3R9+FXgdOSHNKyNi3qREYbro9hdJTkP46vvS7J4S0La8ElHI0kubiq9pzz+vyq+sXx11dW1c+3q06LWUt+F1TVPuOvL6+qmduR3JO15HduVf3SeB3t5X7/TS/Hzr4lubSqdh9/vQlwTlX9yng5zr9W1S+0rVDrkuSSqtpjzusvVNUvJ7kXcFFV7dqwvGXnDHQ730ny/PHHkYfjTuTefDXJ65M8JsmbmfHdyB26OcljAZIcwPjklPGT7DzJYbo5dvbtrvGJKQBbMz71pqp+gN970+72jJ7AS5J9gNsAxht3Z2421sGmnRcBzwA+DTya0RO1AB7E6EB5TbcXMTr7+UjgJ8DLxtfvw+jjSU23/wEcm+QG4H8Ch8Pdm0JdBjDdHDv79ufAhUk+Dfwb8Aa4+3vv4paFaVGvAv45yZcZLZ16Fdyd3SdaFtaCSzgkSdKyGc9A7wRcXVU3tK5Hkxt/0rNlVX23dS2tOQM9Rcb/qlMHkmyc5PeTvCHJY9Z473Wt6tL6Gz9iWB1I8pQkhyXZfo3rL2pVkyZXVd+vqvNWNc9u3O3K/YGfOmkqyR5ruXdFcwa6kSQ3MlozNHfN132AW4CqKh9JO8WSvItRXl9kdHrDOVX1yvF7d28o1HRKcsmal4BHAKseUztz/zPoxbjZeixwAXAAcFxVvXX8nt97Uy7J36x5idEY+h6Aqjpi2YvSRJI8CzgO+C9gU+CFVXXu+L2Z+95zs1M7JzP6V9yrqurbAEn+Y3wUmqbffquarCRvA96e5CPA7+BGmB5cC/wIeCPwY0aZ/SujhkzT7QBg76q6I8lRwN8n2amqXoHfez04CDib0Rr2VXk9Bzi/VUGa2JHAL46fHLkfcGqSI6vqI8zg955LOBqpqsOBvwben+SI8fFZfhzQj81WfVFVd1TVSxidxHEWcL9mVWkiVfUMRptgjgf2rKprgdur6ms+xnvqbVJVdwCMlwAcAGyR5DTmfF9qau3K6AE4TwX+qapOAW6sqlPGX2t6bVxV1wNU1ReBJwCvTXIEM9i/2EA3VFXnA08avzwHuHfDcjTMeUmeOvdCVR0NvBvYoUlFGqSq/i/wNGD/JKdj89WLryZ5/KoXVXVnVR3GaPnNTJ1D26OqurGqXg78FfDeJH+MvUgvblx1jB3AuJneHzgQeFSrolpxDfSUSPJzjD6WPKN1LdKsSbIn8N+q6u9a16KFJdkcoKp+vJb3HlZV/7n8VWl9jE90+ANG33vPb12PFjYeJ2+uqqvXuL4p8Kyqel+bytqwgZYkSZIG8GMTSZIkaQAbaEmSJGkAG2hJkiRpABvoKZDkgoVea7qZX9+SfGKh15peZte38Tne63yt6WV2NtBTYc2n98za03x6Z37de/EirzW9zK5vaz48xYep9GPms/MUjimQZHtgl6r6p/ERTZtU1Y2t69JkzK9v48y2q6qrWteiYcxOUivOQDeW5MXAh4F3ji9tA3y0XUUawvz6luQARk+Q/NT49V7jh6poypld35I8Islnknxp/HqPJK9rXZcWZ3YjNtDt/SHwK8CPAKrqK8BDmlakIcyvb0cB+wE3AFTVRfgkyV4chdn17ATgT4DbAarqEuA5TSvSpMwOG+hp8JOqum3ViySbMIPPlO+Y+fXtjqr6YesitF7Mrm/3qaovrnHtjiaVaCizwwZ6GpyT5Ehg8yS/DpwGfLxxTZqc+fXtS0meC2ycZJckbwU+17ooTcTs+vbdJA9nPOGQ5JnA9W1L0oTMDjcRNpdkI+Aw4MlAgDOr6oS2VWlS5te3JPcBXssoP4AzgTdW1a3tqtIkzK5vSXYCjgceA/wA+A/g+VV1bcu6tDizG7GBbizJocBH557akOQ3q8rzTDtgfn1L8qvA56rqzjnX9qkqz/Kecma3MiS5L7CRJxf1Z9azcwlHe28F/jXJrnOuHd2qGA1mfn07EzgryUPnXHtXq2I0iNl1LMmdSY4BblnVgPkQqj6Y3YgNdHv/AbwI+HCSQ8bX0rAeDWN+fbsK+Evg7CSPGV8zvz6YXd8uY9SDfDrJg8bXzK8PZgds0roAUVV1QZLHA+9P8mhg49ZFaWLm17eqqk8kuQr4YJKT8BSVXphd3+6oqlcneRajT/F+F/PrhdnhDPQ0uB6gqr4LPIXRf4S/0LQiDWF+fQvcfX73rwKPA/ZoWpEmZXZ9W5Xfh4BnAe8GdmpakSZldriJUJLmSbJdVX29dR0azuz6keQXq+r8Oa+3AH6rqt7TsCxNwOxGXMLRSJLjqurlST7OWj76qKpnNChLEzK/viV5dVW9KcnfrOOWI5a1IE3M7PqW5IlVdRawfZLt13j7phY1aTJmN58NdDunjn9+c9MqtL7Mr29XjH8+f8G7NI3Mrm+PB84CDljLewV8ZHnL0QBmN4dLOKZIkgcC246fK6/OmF/fxg/FuV9V/ah1LRrG7CQtNzcRNpbk7CRbjI+CuRh4d5JjW9elyZhf35L8/Ti/+wKXA1cleVXrurQ4s+tbkpeN80uSdyW5IMmTF/+Vas3sRmyg23vAeNbkIODdVfWLwJMa16TJmV/fdhvn91vAGcB2wKFtS9KEzK5vLxrn92TgIcB/B45pW5ImZHbYQE+DTZL8HKOjYHz8c3/Mr2+bJtmUURP2saq6nRk8z7RTZte3VQ/eeDqjyYeLmcGHcXTK7LCBngZHM3ok7dVVdW6SnYCvNK5JkzO/vr0TuBa4L/Av453lrqPtg9n17fwkn2bUhJ2Z5P7AXY1r0mTMDjcRTr0kf1JV/6d1HVo/5teXJAE2rqo7xq9fUFWnNC5LEzC7vow3fu4FXFNVNyTZEnjYqk3YSR5VVZc1LVJrZXYjNtBTLskFVbVP6zq0fsyvb+bXL7Prm/n1a1aycwnH9Ju5dUUrjPn1zfz6ZXZ9M79+zUR2NtDTz48I+mZ+fTO/fpld38yvXzORnQ309JuJf8mtYObXN/Prl9lJWjI20NPvtNYFaIOYX98+27oArTez69ttrQvQepuJ7NxE2FiSHYHDgR2ATVZdr6pntKpJkzO/viX5GeB3+en8jmhVkyZjdv1Lsgc/nd9HmhWkiZndnL+4mvkocCLwcWbwHMUVwPz6dgbwBeBSzK83ZtexJCcBewCXsTq/AmaqCeuR2Y04A91Ykn+vqke3rkPrx/z6NivHLa1EZte3JJdX1W6t69BwZjdiA91YkucCuwCfBn6y6npVXdCsKE3M/PqW5BXATYwewz43v+83K0oTMbu+JTkR+Kuqurx1LRrG7EZcwtHe7sChwBOZ/1HIE5tVpCHMr2+3AX8JvJbVRy8VsFOzijQps+vbKcDnk3yL0T+AAlRV7dG2LE3A7HAGurkkVwJ7VNVM7Fpdacyvb0m+Cjy6qr7buhYNY3Z9S3I18ErWWMNeVV9rVpQmYnYjzkC3dzHwM8B/tS5E68X8+nYZcEvrIrRezK5vX6+q01sXofVidthAT4OHAlcmOZf56/g8Bq0P5te3O4GLkvwz8/PzKLTpZ3Z9uzLJ3zM6wWhufjN1kkOnzA4b6Gnwv1oXoA1ifn376PiH+mN2fducUfP15DnXZu4otE6ZHa6BliRJkgZxBrqRJP9WVY9NciOrd5DD6t2sWzQqTRMwv74l+VBVPSvJpczPD4BZ203eE7PrW5JXV9WbkryVtefnEpwpZXbz2UA3UlWPHf98/9a1aDjz697Lxj//ZtMqtD7Mrm9XjH8+r2kVWh9mN4dLOCRJkqQBNmpdwKxKsuiT6ia5R22YX9+SfOKeuEfLz+z6luSoe+IeLT+zm88Z6EaS/Bj4ykK3AA+oqu2WqSQNYH59S3ID8C8L3QI8qqp8qt2UMbu+JfkmcOxCtwAvrqqfX6aSNCGzm8810O1M8h/YnUtehdaX+fXtwAnu8emS08ns+nYCsNjekROWoxANZnZzOAMtSZIkDeAaaEmSJGkAG2hJkiRpANdAN5LkQRPcdldV3bDkxWgw85MkaXa5BrqRJLcC1zHatbouG3uKw3Qyv74l+dFitwDXV9UjlqMeTc7s+pbk9Alu+35VvXCpa9EwZjefM9DtXFFVey90Q5ILl6sYDWZ+ffuq+XXL7Pq2K/B7C7wf4G+XqRYNY3ZzOAPdSJJ7V9WtG3qP2jC/viXZqaqu2dB7tPzMrm9JnlVVH9rQe7T8zG4+G+gpkuRBVfX91nVo/ZifJEmzwVM4Gknyujlf75bky8D5Sa5N8uiGpWkC5rdyJbm0dQ1aP2bXtyTHt65B62cWs3MGupEkF1TVPuOv/xF4W1V9Msl+wHFV9Zi2FWoh5te3JAet6y3g76pqq+WsR5Mzu74tcIJRgIurapvlrEeTM7v53EQ4Hbauqk8CVNUXk2zeuiANYn79+SDwPmBtMwj3XuZaNIzZ9e07wNeYf4JRjV8/pElFmpTZzWED3c5O4yNhAmyT5D5Vdcv4vU0b1qXJmF/fLgHeXFVfWvONJE9qUI8mZ3Z9uwb4tar6+ppvJPlGg3o0ObObwwa6nQPXeL0xQJKHAu9Y/nI0kPn17eXAus4T/u3lLESDmV3fjgMeCPxUEwa8aZlr0TBmN4droCVJkqQBnIFuJMkDgD8BfgtYtenlv4CPAcf4COjpZn59S7IJcBijGcutGa3ju45RfidW1e0Ny9MCzK5/SX6e0ad4D2N1fqdX1RVNC9OizG41j7Fr50PAD4D9q2rLqtoSeML42mlNK9MkzK9vpwJ7AUcBTwd+A/gzYE/gve3K0gTMrmNJ/ifwAUb7R74InDv++v1JXtOyNi3M7OZzCUcjSa6qqkcOfU/Twfz6tkh+X66qRyx3TZqM2fVtfGb+o9b8pCDJZsBlVbVLm8q0GLObzxnodr6W5NXjTWfAaAPa+F94M7ebtUPm17cfJDkkyd1jYJKNkjyb0acIml5m17e7GC29WdPPjd/T9DK7OVwD3c6zgdcA5yRZdX7it4HTgWc1q0qTMr++PQf4C+DtSX7A6GPInwHOGr+n6WV2fXs58JkkX2H1ZMN2wM7AS5tVpUmY3Rwu4ZA005JsyWgs/G7rWjSM2fVp/OnBfow2ogX4JnBuVd3ZtDAtyuxWcwZ6iiT5RFX9Zus6tH7Mry9r7iZPch3wsaq6sm1lWozZda/m/Lhrzs+afmY35hro6fKw1gVog5hfJxbYTf6BWdxN3hOz61uSJwNf4adPUfnK+D1NKbObzyUcUyTJSVX1otZ1aP2YXz/cTd4vs+tbkiuAp1XVtWtc3xE4o6p2bVKYFmV28zkDPQWSbJ7kkTZffTK/LrmbvF9m17dNGK2bXdN/Apsucy0axuzmcA10Y0kOAN4MbAbsmGQv4OiqekbbyjQJ8+uWu8n7ZXZ9Owk4N8kHWJ3ftoxOUDmxWVWahNnN4RKOxpKcDzwROLuq9h5fu6Sq9mhbmSZhfv1yN3m/zK5vSXZl9SbQVfmdXlWXNy1MizK71ZyBbu+OqvphktZ1aP2YX7/cTd4vs+tYVV0BXNG6Dg1ndqvZQLf3pSTPBTZOsgtwBPC5xjVpcubXofGO8bcz2lH+n+PL2wA7J/mDqvp0s+K0ILNbuZIcVVVHta5Dw81idi7haCzJfYDXAquOgDkTeGNV3dquKk3K/PrkbvJ+md3KleSAqvp46zo03CxmZwMtaeaMN6DtWlV3rHF9M+Dyqtq5TWVajNlJmgYu4ZhCSY6vqpe0rkPrx/y64G7yfpldx5JsAhwG/Daj4wgLuA74GHDimud7a3qY3XzOQDeS5EHregu4uKq2Wc56NIz59c/d5P0yu34leT9wA3AKq88U3gZ4AfCgqnp2q9q0MLObzwa6kSR3Al9jNPivUuPXD6uqzZoUpomYnyQNl+SqqnrkOt77clU9Yrlr0mTMbj6fRNjONcD+VbXjnB87VdWOwLdbF6dFmd8KleSo1jVo/ZhdF36Q5JDxWd7A6FzvJM8GftCwLi3O7OawgW7nOOCB63jvTctZiNaL+a1c57cuQOvN7Kbfc4BnAt9O8uUkXwa+BRw0fk/Ty+zmcAmHJEladkm2ZNSHfLd1LRrG7GygJc0gd5P3y+wkTQMbaEkzx93k/TI7SdPABlrSzHE3eb/MTtI0cBOhpFnkbvJ+mZ2k5mygp0CSCxZ6relmfl1yN3m/zG6FcOzsl9m5hEPSjHM3eb/MTlIrzkBPgSTbJ3nS+OvNk9y/dU2anPn1raq+ZwPWJ7Prm2Nnv8zOBrq5JC8GPgy8c3xpG+Cj7SrSEOYnScM5dvbL7EZsoNv7Q+BXgB8BVNVXgIc0rUhDmJ8kDefY2S+zwwZ6Gvykqm5b9WL8kAAXpvfD/CRpOMfOfpkdNtDT4JwkRwKbJ/l14DTg441r0uTMr3PuJu+X2XXNsbNfZoencDQ3Psv0MODJQIAzq+qEtlVpUuYnScM5dvbL7EZsoBtL8rKq+uvFrmk6mV//kmwP7FJV/5Rkc2CTqrqxdV1anNn1y7GzX2Y34hKO9l6wlmsvXO4itN7Mr2PuJu+X2XXPsbNfZgds0rqAWZXkd4DnAjsmOX3OW/cHvtemKk3K/FaMPwT2A/4dRrvJk8zcbvJOmV2HHDv7ZXbz2UC38zngeuDBwF/NuX4jcEmTijSE+a0MP6mq25IAs7ubvFNm1yfHzn6Z3RyugZY0s5K8CbgB+F3gcOAPgMur6rVNC9OizE5SS66BbizJLyc5N8lNSW5LcmeSH7WuS5Mxv+69BvgOcCnw+8AZNmDdMLuOOXb2y+xGXMLR3tuA5zA6R3FfRrMpOzetSEOYX98OH+8cv/sIplncTd4ps+ubY2e/zA5noKdCVV0NbFxVd1bVu4EntK5JkzO/rrmbvF9m1znHzn6ZnTPQ0+CWJJsBF43X9F0P3LdxTZqc+XXI3eT9MrsVw7GzX2aHmwibGz8I4NvAZsArgAcAbx//605Tzvz6NM5tR+D/MFpLu8qNwCVVdUeTwrQos1sZHDv7ZXYjNtANJdkYOKWqnt+6Fg1nfpI0nGNnv8xuNddAN1RVdwJbjT8KUWfMr3/uJu+X2fXLsbNfZreaa6Dbuxb47Hgt382rLlbVsc0q0hDXYn49czd5v8yub9fi2NmrazE7G+gpcN34x0aMNsGoL+bXuaq6OsnG45mVdyf5XOuaNBmz65pjZ7/MDtdAT70kb62qw1vXofVjftMtyb8ATwLeBXyL0W7yF1bVnk0L06LMbmVz7OzXrGTnGujp9yutC9AGMb/pdiijcfCljD6K3BY4uGlFmpTZrWyOnf2aiexcwiFpJo13k//v8W7yW4E/a1ySJmR2klpzBlrSTHI3eb/MTlJrzkBPv7QuQBvE/KbbtbibvFfXYnYrmWNnv2YiOxvo6ffXrQvQBjG/6eZu8n6Z3crm2NmvmcjOUzgaS/JxYM0QfgicB7yzqm5d/qo0KfNb2WZlN/lKZHbTzbGzX2Y34hro9q4BbgJOGP/4EaNnzD9i/FrTzfxWtpnYTb5Cmd10c+zsl9nhEo5psHdVPW7O648n+ZeqelySy5pVpUmZnyQN59jZL7PDGehpsFWS7Va9GH/94PHL29qUpAHMT5KGc+zsl9nhDPQ0+CPg35J8ldHO1R2BP0hyX+CUppVpEua3ss3EbvIVyuymm2Nnv8wONxFOhST3An6e0X+IV87KAvyVwvxWriQvrKqTW9eh4cxu+jl29svsbKCnQpLHADsw5xOBqnpPs4I0iPn1y93k/TK7/jl29svsXMLRXJJTgYcDFwF3ji8XMFP/IfbK/Lp3DbAV8P7x62czfzf5oY3q0uLMrmOOnf0yuxFnoBtLcgWwWxlEl8yvb6t2jq/tWpLLqupRrWrTwsyub46d/TK7EU/haO9LwM+2LkLrzfz65m7yfpld3xw7+2V2uIRjGjwYuDzJF4GfrLpYVc9oV5IGML++uZu8X2bXN8fOfpkdLuFoLsnj13a9qs5Z7lo0nPn1z93k/TK7fjl29svsRmygJc00d5P3y+wkteISjkaS/FtVPTbJjcw/iilAVdUWjUrTBMxvZXA3eb/Mrk+Onf0yu/mcgZY0s9xN3i+zk9SSM9BTIMnGwEOZ/zHk19tVpCHMr2urdpNf37oQDWZ2nXPs7JfZ2UA3l+Rw4H8xegDAXePLBezRrChNzPy6527yfpldxxw7+2V2Iy7haCzJ1cCjq+p7rWvRcObXN3eT98vs+ubY2S+zG3EGur1vAD9sXYTWm/l1zGarX2bXPcfOfpkdNtDT4Brg7CT/yPyPIY9tV5IGML8OuZu8X2a3Yjh29svssIGeBl8f/9hs/EN9Mb8OVdVjxz/fv3UtGsbsVgzHzn6ZHTbQTY13sd6vql7VuhYNZ34rg7vJ+2V2fXLs7JfZrWYD3VBV3Zlkn9Z1aP2YX//cTd4vs+uXY2e/zG41T+FoLMlfAbsApwE3r7peVR9pVpQmZn59czd5v8yub46d/TK7EWeg23sQ8D3giXOuFTBT/yF2zPz65m7yfpld3xw7+2V2OAMtaYYlORF4JDDTu8l7ZHaSWnIGurEk9wYOAx4F3HvV9ap6UbOiNDHz6567yftldh1z7OyX2Y3YQLd3KnAl8BTgaOB5wBVNK9IQ5tcpd5P3y+xWBMfOfpkdLuFoLsmFVbV3kkuqao8kmwJnVtUTF/3Fas78+pbkM1X1a63r0HBm1zfHzn6Z3Ygz0O3dPv75hiS/AHwL2KFdORrI/Pp2UZLTmfHd5J0yu745dvbL7LCBngbHJ3kg8HrgdOB+wJ+2LUkDmF/f3E3eL7Prm2Nnv8wOl3BIkiRJgzgD3ViShwJ/DmxdVU9Lshvw36rqxMalaQLm1zd3k/fL7Prm2NkvsxvZqHUB4mTgTGDr8esvAy9vVo2GOhnz69mpwM8y2k1+DrANcGPTijQps+vbyTh29upkzM4Gego8uKo+BNwFUFV3AHe2LUkDmF/fdq6q1wM3V9UpwG8AuzeuSZMxu745dvbL7LCBngY3J9mS0eYXkvwyPp62J+bXtzV3kz+AGdxN3imz65tjZ7/MDtdAT4NXMtrF+vAknwW2Ap7ZtiQNYH59czd5v8yub46d/TI7PIVjKiTZBHgkEOCqqrp9kV+iKWJ+kjScY2e/zM4GupkkBy30vg8DmG7mtzK4m7xfZtcnx85+md18NtCNJHn3Am+XRzFNN/NbGZJ8Eng38Nqq2nM8q3JhVbkZbcqZXZ8cO/tldvPZQEuaWUnOrapfSnJhVe09vnZRVe3VujYtzOwkteQpHI0k+c174h61YX4rhrvJ+2V2HXLs7JfZzecMdCNJrgCey2gB/rqcXFV7LFNJGsD8VoYk+wBvBX4B+BLj3eRVdUnTwrQos+uTY2e/zG4+G+hGkpzNeOZkAd+vqoOXoRwNZH4rh7vJ+2V2/XHs7JfZzWcDLWnmuJu8X2YnaRr4IBVJs+iABd4rwCZsepmdpOacgZYkSZIG8BQOSTPH3eT9MjtJ08AZ6CmQ5Ger6lvreq3pZn79cTd5v8xu5XDs7JfZ2UBPhST/WFW/sa7Xmm7m1x93k/fL7FYOx85+mZ0NtCRJkjSIp3A0kuTewP8AdgYuAU6qqjvaVqVJmZ8kDefY2S+zm88Z6EaSfBC4HfhX4GnA16rqZW2r0qTMT5KGc+zsl9nNZwPdSJJLq2r38debAF+sqn0al6UJmZ8kDefY2S+zm89j7Nq5+5Gzs/wRSMfMb4VI8rMLvdb0MrsuOXb2y+zmcAa6kSR3AjeveglsDtwy/rqqaotWtWlx5rdyuJu8X2bXH8fOfpndfDbQkiRJ0gCewiFp5ribvF9mJ2kaOAMtaea4m7xfZidpGthAS5o57ibvl9lJmgaewiFpFrmbvF9mJ6k5Z6AlzRx3k/fL7CRNAxtoSZIkaQCXcEhRMYZsAAAALUlEQVSSJEkD2EBLkiRJA9hAS5IkSQPYQEuSJEkD2EBLkiRJA9hAS5IkSQP8fzdUVk3bZs7ZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_sweeper_df(df.mean(level=(1)), sort_by=\"accuracy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/scrapbook.scrap.json+json": {
       "data": 36,
       "encoder": "json",
       "name": "nr_elements",
       "version": 1
      }
     },
     "metadata": {
      "scrapbook": {
       "data": true,
       "display": false,
       "name": "nr_elements"
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/scrapbook.scrap.json+json": {
       "data": 0.931818,
       "encoder": "json",
       "name": "max_accuray",
       "version": 1
      }
     },
     "metadata": {
      "scrapbook": {
       "data": true,
       "display": false,
       "name": "max_accuray"
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/scrapbook.scrap.json+json": {
       "data": 0.136364,
       "encoder": "json",
       "name": "min_accuray",
       "version": 1
      }
     },
     "metadata": {
      "scrapbook": {
       "data": true,
       "display": false,
       "name": "min_accuray"
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/scrapbook.scrap.json+json": {
       "data": 24.365971,
       "encoder": "json",
       "name": "max_duration",
       "version": 1
      }
     },
     "metadata": {
      "scrapbook": {
       "data": true,
       "display": false,
       "name": "max_duration"
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/scrapbook.scrap.json+json": {
       "data": 15.990457999999999,
       "encoder": "json",
       "name": "min_duration",
       "version": 1
      }
     },
     "metadata": {
      "scrapbook": {
       "data": true,
       "display": false,
       "name": "min_duration"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Preserve some of the notebook outputs\n",
    "sb.glue(\"nr_elements\", len(df))\n",
    "sb.glue(\"max_accuray\", df.max().accuracy)\n",
    "sb.glue(\"min_accuray\", df.min().accuracy)\n",
    "sb.glue(\"max_duration\", df.max().duration)\n",
    "sb.glue(\"min_duration\", df.min().duration)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (cv)",
   "language": "python",
   "name": "cv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
